[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:04 dist-test-slave-vxj2 krb5kdc[16300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:04 dist-test-slave-vxj2 krb5kdc[16300](info): set up 2 sockets
Aug 10 06:45:04 dist-test-slave-vxj2 krb5kdc[16300](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 06:45:05.118945 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:05 dist-test-slave-vxj2 krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:45:05.248808 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:35967
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:35967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:05.333904 16315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:05.334180 16315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:05.334221 16315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:05.335431 16315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:05.335515 16315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:05.335539 16315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:05.335569 16315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:05.335600 16315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:05.337204 16315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:35967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:35967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:05.337500 16315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:05.337752 16315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:45:05.343640 16315 server_base.cc:1034] running on GCE node
W20260810 06:45:05.343621 16326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:05.343631 16320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:05.343629 16321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:05.344131 16315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:05.344487 16315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:05.345664 16315 hybrid_clock.cc:648] HybridClock initialized: now 1786344305345661 us; error 109 us; skew 500 ppm
Aug 10 06:45:05 dist-test-slave-vxj2 krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:05.348269 16315 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:45:05.348762 16315 webserver.cc:460] Webserver started at http://127.15.232.254:46203/ using document root <none> and password file <none>
I20260810 06:45:05.348963 16315 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:05.349004 16315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:05.349100 16315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:05.350270 16315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "ad57cbaf589b4884a4987cc9e92e6967"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-vxj2"
server_key: "677678ce8e0f6b1a0e8f0fad2dc9fa9b"
server_key_iv: "ccde644d62b9e48dff6d403a59239992"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.350579 16315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "ad57cbaf589b4884a4987cc9e92e6967"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-vxj2"
server_key: "677678ce8e0f6b1a0e8f0fad2dc9fa9b"
server_key_iv: "ccde644d62b9e48dff6d403a59239992"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.351930 16315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:05.352623 16333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.352821 16315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:05.352924 16315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "ad57cbaf589b4884a4987cc9e92e6967"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-vxj2"
server_key: "677678ce8e0f6b1a0e8f0fad2dc9fa9b"
server_key_iv: "ccde644d62b9e48dff6d403a59239992"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.352981 16315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:05.376865 16315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:05.379678 16315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:05.379819 16315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:05.385784 16315 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:35967
I20260810 06:45:05.385823 16399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:35967 every 8 connection(s)
I20260810 06:45:05.386339 16315 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:05.387661 16400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:05.391472 16400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967: Bootstrap starting.
I20260810 06:45:05.392184 16400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:05.392452 16400 log.cc:824] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:05.393322 16400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967: No bootstrap required, opened a new log
I20260810 06:45:05.394976 16400 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad57cbaf589b4884a4987cc9e92e6967" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35967 } }
I20260810 06:45:05.395069 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 16315
I20260810 06:45:05.395134 16400 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:05.395162 16400 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad57cbaf589b4884a4987cc9e92e6967, State: Initialized, Role: FOLLOWER
I20260810 06:45:05.395205 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:05.395363 16400 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad57cbaf589b4884a4987cc9e92e6967" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35967 } }
I20260810 06:45:05.395452 16400 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:05.395499 16400 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:05.395555 16400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:05.396046 16291 external_mini_cluster.cc:1428] Setting key 4d5c52e4a425413024a5258707e3d0b1
I20260810 06:45:05.396206 16400 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad57cbaf589b4884a4987cc9e92e6967" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35967 } }
I20260810 06:45:05.396345 16400 leader_election.cc:302] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad57cbaf589b4884a4987cc9e92e6967; no voters:
I20260810 06:45:05.396560 16400 leader_election.cc:288] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:05.396620 16404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:05.396806 16404 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [term 1 LEADER]: Becoming Leader. State: Replica: ad57cbaf589b4884a4987cc9e92e6967, State: Running, Role: LEADER
I20260810 06:45:05.396885 16400 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:05.397053 16404 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad57cbaf589b4884a4987cc9e92e6967" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35967 } }
I20260810 06:45:05.397598 16404 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad57cbaf589b4884a4987cc9e92e6967. Latest consensus state: current_term: 1 leader_uuid: "ad57cbaf589b4884a4987cc9e92e6967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad57cbaf589b4884a4987cc9e92e6967" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35967 } } }
I20260810 06:45:05.398557 16404 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:05.398845 16407 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad57cbaf589b4884a4987cc9e92e6967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad57cbaf589b4884a4987cc9e92e6967" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35967 } } }
I20260810 06:45:05.399166 16425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:05.399214 16407 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:05.399711 16425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:05.401608 16425 catalog_manager.cc:1347] Generated new cluster ID: 3bda03ac3d474945a543e931d4b1ea3b
I20260810 06:45:05.401675 16425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:05.408510 16425 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:05.409304 16425 catalog_manager.cc:1504] Loading token signing keys...
Aug 10 06:45:05 dist-test-slave-vxj2 krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:05.415386 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.397832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50242 (local address 127.15.232.254:35967)
0810 06:45:05.398068 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.398079 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.398121 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:45:05.400644 (+ 2523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.400653 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.400735 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.400967 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:05.403269 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.404343 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.405678 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.406176 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.406325 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.410974 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:05.410979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:05.410979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:05.410996 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:05.412967 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.413572 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.413576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.413576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.413676 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.414002 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.414003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.414004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.414279 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:05.414480 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.414805 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.415061 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":117,"threads_started":1}
I20260810 06:45:05.421355 16425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad57cbaf589b4884a4987cc9e92e6967: Generated new TSK 0
I20260810 06:45:05.421590 16425 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:45:05.461066 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35967
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:05.563297 16430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:05.563561 16430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:05.563609 16430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:05.564790 16430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:05.564862 16430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:05.564914 16430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:05.566537 16430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35967
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:05.566874 16430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:05.567131 16430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:05.573585 16439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:05.573676 16435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:05.573592 16436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:05.574102 16430 server_base.cc:1034] running on GCE node
I20260810 06:45:05.574364 16430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:05.574640 16430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:05.575839 16430 hybrid_clock.cc:648] HybridClock initialized: now 1786344305575827 us; error 35 us; skew 500 ppm
Aug 10 06:45:05 dist-test-slave-vxj2 krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:05.578693 16430 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:45:05.579248 16430 webserver.cc:460] Webserver started at http://127.15.232.193:37175/ using document root <none> and password file <none>
I20260810 06:45:05.579514 16430 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:05.579582 16430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:05.579730 16430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:05.580799 16430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "60febeec2b5640c1b08b4e56adfa27d7"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-vxj2"
server_key: "767401104dcd7ee075f050d6bcae2bd5"
server_key_iv: "657fe058485dba16844867a59788a139"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.581224 16430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "60febeec2b5640c1b08b4e56adfa27d7"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-vxj2"
server_key: "767401104dcd7ee075f050d6bcae2bd5"
server_key_iv: "657fe058485dba16844867a59788a139"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.582656 16430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:05.583464 16449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.583730 16430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:05.583844 16430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "60febeec2b5640c1b08b4e56adfa27d7"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-vxj2"
server_key: "767401104dcd7ee075f050d6bcae2bd5"
server_key_iv: "657fe058485dba16844867a59788a139"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.583945 16430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:05.597375 16430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:05.599867 16430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:05.600047 16430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:05.600303 16430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:05.600644 16430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:05.600703 16430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.600747 16430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:05.600775 16430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.607424 16430 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:39323
I20260810 06:45:05.607470 16581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:39323 every 8 connection(s)
I20260810 06:45:05.607808 16430 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
Aug 10 06:45:05 dist-test-slave-vxj2 krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:05.617297 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 16430
I20260810 06:45:05.617426 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:05.617600 16291 external_mini_cluster.cc:1428] Setting key 5c5e2b3a67e754ca5fda7afc968401ff
I20260810 06:45:05.617676 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.608558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:55783 (local address 127.15.232.254:35967)
0810 06:45:05.608621 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.608623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.608867 (+ 244us) server_negotiation.cc:407] Connection header received
0810 06:45:05.609371 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.609372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.609383 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.609449 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:05.610593 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.611108 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.612127 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.612325 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.612406 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.614744 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:05.614745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:05.614746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:05.614764 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:05.616156 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.616541 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.616542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.616542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.616598 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.616871 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.616872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.616873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.617004 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:05.617087 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.617477 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.617580 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:45:05.617866 16584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.608510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35967 (local address 127.15.232.193:55783)
0810 06:45:05.608694 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:05.608748 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:05.609259 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:05.609536 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:05.609539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:05.610290 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:05.610497 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.610502 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.611201 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.611202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.612034 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.612036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.612439 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.612441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.612560 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.613144 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:05.613170 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:05.614586 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:05.616236 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:05.616242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:05.616244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:05.616470 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:05.616663 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:05.616664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:05.616664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:05.616785 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:05.617093 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:05.617102 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:05.617295 (+ 193us) client_negotiation.cc:769] Sending connection context
0810 06:45:05.617425 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 06:45:05.617602 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":99,"threads_started":1}
I20260810 06:45:05.618407 16582 heartbeater.cc:347] Connected to a master server at 127.15.232.254:35967
I20260810 06:45:05.618520 16582 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:05.618721 16582 heartbeater.cc:511] Master 127.15.232.254:35967 requested a full tablet report, sending...
I20260810 06:45:05.619266 16352 ts_manager.cc:194] Registered new tserver with Master: 60febeec2b5640c1b08b4e56adfa27d7 (127.15.232.193:39323)
I20260810 06:45:05.620123 16352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:55783
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:45:05.645727 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35967
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:05.728930 16587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:05.729228 16587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:05.729286 16587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:05.730465 16587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:05.730537 16587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:05.730587 16587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:05.732101 16587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35967
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:05.732378 16587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:05.732630 16587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:45:05.738065 16587 server_base.cc:1034] running on GCE node
W20260810 06:45:05.738031 16593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:05.738031 16596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:05.738077 16594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:05.738483 16587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:05.738742 16587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:05.739920 16587 hybrid_clock.cc:648] HybridClock initialized: now 1786344305739929 us; error 53 us; skew 500 ppm
Aug 10 06:45:05 dist-test-slave-vxj2 krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:05.742571 16587 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:45:05.743148 16587 webserver.cc:460] Webserver started at http://127.15.232.194:46349/ using document root <none> and password file <none>
I20260810 06:45:05.743387 16587 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:05.743434 16587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:05.743597 16587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:05.744716 16587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "8eb1ac0df7b64a2ab0bda4c55ac90893"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-vxj2"
server_key: "c2898007c03b6af099f55616708d8bce"
server_key_iv: "abefdeaec4012cd9fa2620c41dcc3e46"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.745116 16587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "8eb1ac0df7b64a2ab0bda4c55ac90893"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-vxj2"
server_key: "c2898007c03b6af099f55616708d8bce"
server_key_iv: "abefdeaec4012cd9fa2620c41dcc3e46"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.746563 16587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:05.747377 16605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.747560 16587 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:45:05.747663 16587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "8eb1ac0df7b64a2ab0bda4c55ac90893"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-vxj2"
server_key: "c2898007c03b6af099f55616708d8bce"
server_key_iv: "abefdeaec4012cd9fa2620c41dcc3e46"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.747749 16587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:05.786051 16587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:05.788544 16587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:05.788730 16587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:05.789007 16587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:05.789377 16587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:05.789438 16587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.789472 16587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:05.789520 16587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.795810 16587 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:35991
I20260810 06:45:05.795885 16740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:35991 every 8 connection(s)
I20260810 06:45:05.796239 16587 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:05.801309 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 16587
I20260810 06:45:05.801424 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:05.801975 16291 external_mini_cluster.cc:1428] Setting key e8a3aa2dea1140dab3df7c3c5aa7a1e4
Aug 10 06:45:05 dist-test-slave-vxj2 krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:05.807173 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.797013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:51577 (local address 127.15.232.254:35967)
0810 06:45:05.797105 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.797107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.797379 (+ 272us) server_negotiation.cc:407] Connection header received
0810 06:45:05.798048 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.798049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.798060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.798118 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:05.799263 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.799755 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.800776 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.800979 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.801033 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.803861 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:05.803863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:05.803863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:05.803874 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:05.805672 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.806128 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.806130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.806131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.806198 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.806470 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.806471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.806471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.806598 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:05.806673 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.806973 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.807083 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:45:05.807379 16743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.796963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35967 (local address 127.15.232.194:51577)
0810 06:45:05.797233 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:05.797282 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:05.797907 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:05.798175 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:05.798178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:05.798919 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:05.799174 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.799180 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.799869 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.799870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.800700 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.800703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.801056 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.801057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.801253 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.801922 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:05.801934 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:05.803728 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:05.805769 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:05.805777 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:05.805779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:05.806026 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:05.806275 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:05.806276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:05.806276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:05.806388 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:05.806678 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:05.806685 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:05.806831 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 06:45:05.806940 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 06:45:05.807115 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":114,"threads_started":1}
I20260810 06:45:05.808050 16741 heartbeater.cc:347] Connected to a master server at 127.15.232.254:35967
I20260810 06:45:05.808151 16741 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:05.808370 16741 heartbeater.cc:511] Master 127.15.232.254:35967 requested a full tablet report, sending...
I20260810 06:45:05.808797 16352 ts_manager.cc:194] Registered new tserver with Master: 8eb1ac0df7b64a2ab0bda4c55ac90893 (127.15.232.194:35991)
I20260810 06:45:05.809242 16352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:51577
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:45:05.831992 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35967
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:05.917294 16746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:05.917517 16746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:05.917543 16746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:05.918677 16746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:05.918718 16746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:05.918752 16746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:05.920246 16746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35967
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:05.920451 16746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:05.920635 16746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:05.926116 16752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:05.926127 16751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:05.926122 16756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:05.926436 16746 server_base.cc:1034] running on GCE node
I20260810 06:45:05.926667 16746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:05.926934 16746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:05.928085 16746 hybrid_clock.cc:648] HybridClock initialized: now 1786344305928073 us; error 34 us; skew 500 ppm
Aug 10 06:45:05 dist-test-slave-vxj2 krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:05.930706 16746 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:45:05.931185 16746 webserver.cc:460] Webserver started at http://127.15.232.195:40687/ using document root <none> and password file <none>
I20260810 06:45:05.931413 16746 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:05.931461 16746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:05.931592 16746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:05.932691 16746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "8c665cfae0024f2ca37a88bbeaffa876"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-vxj2"
server_key: "5e437475f559bc78699a6d0edfd33b81"
server_key_iv: "8ede99ddd48d6d43b12ae01cc4511aaf"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.933070 16746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "8c665cfae0024f2ca37a88bbeaffa876"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-vxj2"
server_key: "5e437475f559bc78699a6d0edfd33b81"
server_key_iv: "8ede99ddd48d6d43b12ae01cc4511aaf"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.934564 16746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:05.935420 16763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.935635 16746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:45:05.935730 16746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "8c665cfae0024f2ca37a88bbeaffa876"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-vxj2"
server_key: "5e437475f559bc78699a6d0edfd33b81"
server_key_iv: "8ede99ddd48d6d43b12ae01cc4511aaf"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.935809 16746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:05.949090 16746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:05.951399 16746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:05.951562 16746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:05.951817 16746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:05.952149 16746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:05.952205 16746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.952248 16746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:05.952277 16746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.958812 16746 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:40817
I20260810 06:45:05.958858 16897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:40817 every 8 connection(s)
I20260810 06:45:05.959208 16746 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:45:05 dist-test-slave-vxj2 krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:05.967547 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 16746
I20260810 06:45:05.967649 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:05.967893 16291 external_mini_cluster.cc:1428] Setting key 74695e5fdf73965243b04724f5f911ab
I20260810 06:45:05.969019 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.960042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:40129 (local address 127.15.232.254:35967)
0810 06:45:05.960110 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.960112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.960361 (+ 249us) server_negotiation.cc:407] Connection header received
0810 06:45:05.960836 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.960836 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.960846 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.960928 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:05.961907 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.962390 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.963372 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.963544 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.963592 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.965802 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:05.965803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:05.965803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:05.965809 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:05.967273 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.967816 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.967817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.967818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.967889 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.968204 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.968205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.968205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.968346 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:05.968428 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.968780 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.968916 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:45:05.969213 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.960044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35967 (local address 127.15.232.195:40129)
0810 06:45:05.960217 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:05.960261 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:05.960755 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:05.960993 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:05.960996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:05.961604 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:05.961812 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.961817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.962477 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.962478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.963290 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.963293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.963614 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.963615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.963718 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.964270 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:05.964279 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:05.965725 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:05.967403 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:05.967410 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:05.967412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:05.967743 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:05.967965 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:05.967966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:05.967967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:05.968104 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:05.968420 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:05.968428 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:05.968596 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 06:45:05.968732 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 06:45:05.968930 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260810 06:45:05.969718 16898 heartbeater.cc:347] Connected to a master server at 127.15.232.254:35967
I20260810 06:45:05.969831 16898 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:05.970016 16898 heartbeater.cc:511] Master 127.15.232.254:35967 requested a full tablet report, sending...
I20260810 06:45:05.970412 16352 ts_manager.cc:194] Registered new tserver with Master: 8c665cfae0024f2ca37a88bbeaffa876 (127.15.232.195:40817)
I20260810 06:45:05.970754 16352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:40129
I20260810 06:45:05.980265 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:45:05 dist-test-slave-vxj2 krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:45:05 dist-test-slave-vxj2 krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.232.193@KRBTEST.COM
I20260810 06:45:06.006042 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.997719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52408 (local address 127.15.232.193:39323)
0810 06:45:05.997936 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.997941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.997978 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:45:05.998009 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.998011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.998026 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.998091 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:05.998826 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.999573 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:06.000303 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:06.000639 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:06.000699 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:06.003146 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:06.003154 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:06.003155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:06.003161 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:06.004894 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:06.005296 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:06.005299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:06.005300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:06.005360 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:06.005540 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:06.005540 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:06.005540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:06.005672 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:06.005783 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:06.005841 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 06:45:06.005906 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":116,"threads_started":1}
W20260810 06:45:06.006448 16479 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52408
Aug 10 06:45:06 dist-test-slave-vxj2 krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:45:06 dist-test-slave-vxj2 krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.193@KRBTEST.COM
I20260810 06:45:06.030598 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:06.022748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52418 (local address 127.15.232.193:39323)
0810 06:45:06.022955 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:06.022958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:06.023045 (+ 87us) server_negotiation.cc:407] Connection header received
0810 06:45:06.023146 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:06.023147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:06.023168 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:06.023236 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:06.023862 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:06.024503 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:06.025315 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:06.025541 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:06.025630 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:06.027962 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:06.027963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:06.027964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:06.027971 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:06.029412 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:06.029784 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:06.029786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:06.029787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:06.029851 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:06.030064 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:06.030065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:06.030065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:06.030169 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:06.030292 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:06.030385 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 06:45:06.030484 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260810 06:45:06.031252 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 16430
I20260810 06:45:06.039611 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 16587
I20260810 06:45:06.049146 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 16746
I20260810 06:45:06.055742 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 16315
2026-08-10T06:45:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2083 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:06 dist-test-slave-vxj2 krb5kdc[16915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:06 dist-test-slave-vxj2 krb5kdc[16915](info): set up 2 sockets
Aug 10 06:45:06 dist-test-slave-vxj2 krb5kdc[16915](info): commencing operation
krb5kdc: starting...
W20260810 06:45:07.188997 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:07 dist-test-slave-vxj2 krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:45:07.313198 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:43437
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:43437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:07.398957 16928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:07.399173 16928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:07.399197 16928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:07.400411 16928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:07.400466 16928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:07.400480 16928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:07.400493 16928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:07.400506 16928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:07.402227 16928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:43437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:43437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:07.402521 16928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:07.402779 16928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:07.408449 16934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:07.408457 16933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:07.408695 16928 server_base.cc:1034] running on GCE node
W20260810 06:45:07.408780 16936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:07.409092 16928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:07.409451 16928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:07.410612 16928 hybrid_clock.cc:648] HybridClock initialized: now 1786344307410585 us; error 46 us; skew 500 ppm
Aug 10 06:45:07 dist-test-slave-vxj2 krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344307, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:07.413360 16928 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:45:07.413805 16928 webserver.cc:460] Webserver started at http://127.15.232.254:36113/ using document root <none> and password file <none>
I20260810 06:45:07.414045 16928 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:07.414121 16928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:07.414276 16928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:07.415411 16928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "fc7b6c4d998a41d5a52c5ce038c1d39b"
format_stamp: "Formatted at 2026-08-10 06:45:07 on dist-test-slave-vxj2"
server_key: "e32a3d58545444c33f142afca0205be4"
server_key_iv: "cb3f5e84a43d0b71f3ede0435d2d472d"
server_key_version: "encryptionkey@0"
I20260810 06:45:07.415787 16928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "fc7b6c4d998a41d5a52c5ce038c1d39b"
format_stamp: "Formatted at 2026-08-10 06:45:07 on dist-test-slave-vxj2"
server_key: "e32a3d58545444c33f142afca0205be4"
server_key_iv: "cb3f5e84a43d0b71f3ede0435d2d472d"
server_key_version: "encryptionkey@0"
I20260810 06:45:07.417277 16928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:07.418061 16943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:07.418234 16928 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:07.418360 16928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "fc7b6c4d998a41d5a52c5ce038c1d39b"
format_stamp: "Formatted at 2026-08-10 06:45:07 on dist-test-slave-vxj2"
server_key: "e32a3d58545444c33f142afca0205be4"
server_key_iv: "cb3f5e84a43d0b71f3ede0435d2d472d"
server_key_version: "encryptionkey@0"
I20260810 06:45:07.418439 16928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:07.431915 16928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:07.434396 16928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:07.434573 16928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:07.438812 16928 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:43437
I20260810 06:45:07.438851 17005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:43437 every 8 connection(s)
I20260810 06:45:07.439221 16928 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:07.440116 17006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:07.442548 17006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b: Bootstrap starting.
I20260810 06:45:07.443159 17006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:07.443456 17006 log.cc:824] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:07.444301 17006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b: No bootstrap required, opened a new log
I20260810 06:45:07.446127 17006 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7b6c4d998a41d5a52c5ce038c1d39b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43437 } }
I20260810 06:45:07.446283 17006 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:07.446310 17006 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc7b6c4d998a41d5a52c5ce038c1d39b, State: Initialized, Role: FOLLOWER
I20260810 06:45:07.446527 17006 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7b6c4d998a41d5a52c5ce038c1d39b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43437 } }
I20260810 06:45:07.446604 17006 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:07.446650 17006 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:07.446714 17006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:07.447320 17006 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7b6c4d998a41d5a52c5ce038c1d39b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43437 } }
I20260810 06:45:07.447417 17006 leader_election.cc:302] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc7b6c4d998a41d5a52c5ce038c1d39b; no voters:
I20260810 06:45:07.447626 17006 leader_election.cc:288] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:07.447667 17009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:07.447849 17009 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [term 1 LEADER]: Becoming Leader. State: Replica: fc7b6c4d998a41d5a52c5ce038c1d39b, State: Running, Role: LEADER
I20260810 06:45:07.447961 17006 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:07.447958 17009 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7b6c4d998a41d5a52c5ce038c1d39b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43437 } }
I20260810 06:45:07.448400 17011 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc7b6c4d998a41d5a52c5ce038c1d39b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7b6c4d998a41d5a52c5ce038c1d39b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43437 } } }
I20260810 06:45:07.448681 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 16928
I20260810 06:45:07.448637 17012 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc7b6c4d998a41d5a52c5ce038c1d39b. Latest consensus state: current_term: 1 leader_uuid: "fc7b6c4d998a41d5a52c5ce038c1d39b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7b6c4d998a41d5a52c5ce038c1d39b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43437 } } }
I20260810 06:45:07.448774 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:07.448789 17012 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:07.449024 16291 external_mini_cluster.cc:1428] Setting key c90017727e7e6ee9153e00d68a0a71ce
I20260810 06:45:07.448485 17011 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b [sys.catalog]: This master's current role is: LEADER
W20260810 06:45:07.450047 17027 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:45:07.450121 17027 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:45:07.450184 17020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:07.450377 17020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:07.452205 17020 catalog_manager.cc:1347] Generated new cluster ID: 84a56afc32094665ad097ef17bf7e156
I20260810 06:45:07.452262 17020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:45:07 dist-test-slave-vxj2 krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:07.461377 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:07.449553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.15.232.254:43437)
0810 06:45:07.449943 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:07.449955 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:07.449987 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:45:07.450141 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:07.450148 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:07.451115 (+ 967us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:07.451269 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:07.452377 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:07.453515 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:07.454353 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:07.454876 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:07.455018 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:07.457595 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:07.457603 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:07.457603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:07.457622 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:07.459505 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:07.459939 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:07.459943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:07.459944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:07.460068 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:07.460324 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:07.460325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:07.460326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:07.460602 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:07.460802 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:07.460912 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 06:45:07.461083 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":70,"threads_started":1}
I20260810 06:45:07.473271 17020 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:07.473919 17020 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:07.478396 17020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc7b6c4d998a41d5a52c5ce038c1d39b: Generated new TSK 0
I20260810 06:45:07.478597 17020 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:45:07.510242 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:43437
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:07.623044 17035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:07.623247 17035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:07.623272 17035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:07.624447 17035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:07.624492 17035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:07.624518 17035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:07.626113 17035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:43437
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:07.626386 17035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:07.626654 17035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:45:07.632081 17035 server_base.cc:1034] running on GCE node
W20260810 06:45:07.632087 17041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:07.632112 17040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:07.632237 17043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:07.632469 17035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:07.632674 17035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:07.633810 17035 hybrid_clock.cc:648] HybridClock initialized: now 1786344307633788 us; error 39 us; skew 500 ppm
Aug 10 06:45:07 dist-test-slave-vxj2 krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344307, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:07.636595 17035 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:45:07.637042 17035 webserver.cc:460] Webserver started at http://127.15.232.193:40657/ using document root <none> and password file <none>
I20260810 06:45:07.637346 17035 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:07.637398 17035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:07.637501 17035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:07.638761 17035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5"
format_stamp: "Formatted at 2026-08-10 06:45:07 on dist-test-slave-vxj2"
server_key: "440b3fdf5546b4b303b5653e66dbde89"
server_key_iv: "def15b09e5a2bc726d0ed7d049d6fcff"
server_key_version: "encryptionkey@0"
I20260810 06:45:07.639091 17035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5"
format_stamp: "Formatted at 2026-08-10 06:45:07 on dist-test-slave-vxj2"
server_key: "440b3fdf5546b4b303b5653e66dbde89"
server_key_iv: "def15b09e5a2bc726d0ed7d049d6fcff"
server_key_version: "encryptionkey@0"
I20260810 06:45:07.640470 17035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:07.641319 17054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:07.641474 17035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:07.641558 17035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5"
format_stamp: "Formatted at 2026-08-10 06:45:07 on dist-test-slave-vxj2"
server_key: "440b3fdf5546b4b303b5653e66dbde89"
server_key_iv: "def15b09e5a2bc726d0ed7d049d6fcff"
server_key_version: "encryptionkey@0"
I20260810 06:45:07.641614 17035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:07.663235 17035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:07.665827 17035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:07.666006 17035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:07.666289 17035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:07.666703 17035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:07.666750 17035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:07.666776 17035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:07.666828 17035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:07.673560 17035 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:43531
I20260810 06:45:07.673658 17181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:43531 every 8 connection(s)
I20260810 06:45:07.674052 17035 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:07.676774 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 17035
I20260810 06:45:07.676862 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:07.677132 16291 external_mini_cluster.cc:1428] Setting key 6e2115f57f6c9e99299f4f144cf1f4a3
Aug 10 06:45:07 dist-test-slave-vxj2 krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344307, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:07.685376 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:07.674918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:41979 (local address 127.15.232.254:43437)
0810 06:45:07.674992 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:07.674995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:07.675243 (+ 248us) server_negotiation.cc:407] Connection header received
0810 06:45:07.676228 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:07.676229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:07.676243 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:07.676309 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:07.677062 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:07.677894 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:07.678977 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:07.679313 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:07.679425 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:07.682057 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:07.682058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:07.682059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:07.682067 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:07.683679 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:07.684256 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:07.684258 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:07.684258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:07.684331 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:07.684620 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:07.684622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:07.684622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:07.684766 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:07.684852 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:07.685167 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 06:45:07.685277 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
I20260810 06:45:07.685592 17184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:07.674914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:43437 (local address 127.15.232.193:41979)
0810 06:45:07.675117 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:07.675156 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:07.676123 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:07.676383 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:07.676386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:07.676665 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:07.676926 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:07.676933 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:07.677998 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:07.677999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:07.678886 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:07.678888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:07.679438 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:07.679440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:07.679581 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:07.680200 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:07.680209 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:07.681949 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:07.683868 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:07.683877 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:07.683880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:07.684145 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:07.684416 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:07.684417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:07.684417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:07.684541 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:07.684844 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:07.684851 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:07.685002 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 06:45:07.685125 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 06:45:07.685325 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":115,"threads_started":1}
I20260810 06:45:07.686089 17182 heartbeater.cc:347] Connected to a master server at 127.15.232.254:43437
I20260810 06:45:07.686213 17182 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:07.686414 17182 heartbeater.cc:511] Master 127.15.232.254:43437 requested a full tablet report, sending...
I20260810 06:45:07.687111 16962 ts_manager.cc:194] Registered new tserver with Master: c1ab0f665b594b0fb3cd1df8f0dc34b5 (127.15.232.193:43531)
I20260810 06:45:07.687891 16962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:41979
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:45:07.703938 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:43437
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:07.789063 17187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:07.789394 17187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:07.789443 17187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:07.790833 17187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:07.790906 17187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:07.790961 17187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:07.792696 17187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:43437
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:07.793020 17187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:07.793362 17187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:07.799029 17196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:07.799029 17192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:07.799029 17193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:07.799396 17187 server_base.cc:1034] running on GCE node
I20260810 06:45:07.799636 17187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:07.799912 17187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:07.801084 17187 hybrid_clock.cc:648] HybridClock initialized: now 1786344307801074 us; error 44 us; skew 500 ppm
Aug 10 06:45:07 dist-test-slave-vxj2 krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344307, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:07.803848 17187 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:45:07.804347 17187 webserver.cc:460] Webserver started at http://127.15.232.194:45751/ using document root <none> and password file <none>
I20260810 06:45:07.804628 17187 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:07.804708 17187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:07.804867 17187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:07.806169 17187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "93f3c9e0a5644000ab45893afeb7851a"
format_stamp: "Formatted at 2026-08-10 06:45:07 on dist-test-slave-vxj2"
server_key: "f6e063bf6303644bddad8f6ffc8336bb"
server_key_iv: "c9a2f646a19e577b2acda93825d8075d"
server_key_version: "encryptionkey@0"
I20260810 06:45:07.806564 17187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "93f3c9e0a5644000ab45893afeb7851a"
format_stamp: "Formatted at 2026-08-10 06:45:07 on dist-test-slave-vxj2"
server_key: "f6e063bf6303644bddad8f6ffc8336bb"
server_key_iv: "c9a2f646a19e577b2acda93825d8075d"
server_key_version: "encryptionkey@0"
I20260810 06:45:07.807986 17187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:45:07.808866 17203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:07.809266 17187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:07.809383 17187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "93f3c9e0a5644000ab45893afeb7851a"
format_stamp: "Formatted at 2026-08-10 06:45:07 on dist-test-slave-vxj2"
server_key: "f6e063bf6303644bddad8f6ffc8336bb"
server_key_iv: "c9a2f646a19e577b2acda93825d8075d"
server_key_version: "encryptionkey@0"
I20260810 06:45:07.809480 17187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:07.829735 17187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:07.832223 17187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:07.832397 17187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:07.832659 17187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:07.833006 17187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:07.833065 17187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:07.833112 17187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:07.833137 17187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:07.839614 17187 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:34109
I20260810 06:45:07.839659 17329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:34109 every 8 connection(s)
I20260810 06:45:07.840059 17187 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:45:07 dist-test-slave-vxj2 krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344307, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:07.849514 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 17187
I20260810 06:45:07.849591 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:07.849761 16291 external_mini_cluster.cc:1428] Setting key dcca499549294e61f787a545d6a91c91
I20260810 06:45:07.851315 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:07.840906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:40615 (local address 127.15.232.254:43437)
0810 06:45:07.841000 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:07.841002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:07.841203 (+ 201us) server_negotiation.cc:407] Connection header received
0810 06:45:07.842238 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:07.842240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:07.842253 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:07.842356 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:07.843044 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:07.843575 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:07.844870 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:07.845083 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:07.845199 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:07.847681 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:07.847684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:07.847684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:07.847695 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:07.849341 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:07.849981 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:07.849982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:07.849983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:07.850063 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:07.850526 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:07.850527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:07.850528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:07.850696 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:07.850781 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:07.851074 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:45:07.851170 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:45:07.851392 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:07.840859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:43437 (local address 127.15.232.194:40615)
0810 06:45:07.841031 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:07.841073 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:07.842090 (+ 1017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:07.842434 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:07.842438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:07.842703 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:07.842912 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:07.842916 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:07.843791 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:07.843793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:07.844758 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:07.844762 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:07.845206 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:07.845208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:07.845374 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:07.845896 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:07.845910 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:07.847573 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:07.849445 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:07.849454 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:07.849456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:07.849873 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:07.850158 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:07.850159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:07.850159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:07.850327 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:07.850792 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:07.850798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:07.850926 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 06:45:07.851032 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 06:45:07.851173 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260810 06:45:07.851866 17330 heartbeater.cc:347] Connected to a master server at 127.15.232.254:43437
I20260810 06:45:07.851989 17330 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:07.852178 17330 heartbeater.cc:511] Master 127.15.232.254:43437 requested a full tablet report, sending...
I20260810 06:45:07.852595 16962 ts_manager.cc:194] Registered new tserver with Master: 93f3c9e0a5644000ab45893afeb7851a (127.15.232.194:34109)
I20260810 06:45:07.852994 16962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:40615
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:45:07.878083 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:43437
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:07.961952 17337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:07.962160 17337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:07.962185 17337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:07.963472 17337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:07.963519 17337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:07.963547 17337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:07.965066 17337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:43437
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:07.965361 17337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:07.965605 17337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:07.970851 17345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:07.970849 17347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:07.971051 17337 server_base.cc:1034] running on GCE node
W20260810 06:45:07.971105 17344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:07.971387 17337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:07.971642 17337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:07.972800 17337 hybrid_clock.cc:648] HybridClock initialized: now 1786344307972774 us; error 44 us; skew 500 ppm
Aug 10 06:45:07 dist-test-slave-vxj2 krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344307, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:07.975718 17337 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:45:07.976224 17337 webserver.cc:460] Webserver started at http://127.15.232.195:38583/ using document root <none> and password file <none>
I20260810 06:45:07.976485 17337 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:07.976555 17337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:07.976701 17337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:07.977852 17337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "5dc99fc597ad4b9cb8736172eb5b7b01"
format_stamp: "Formatted at 2026-08-10 06:45:07 on dist-test-slave-vxj2"
server_key: "69b0137cab75cbd76005c364c3476b9f"
server_key_iv: "23eb742ffdd461b8003d39539550314e"
server_key_version: "encryptionkey@0"
I20260810 06:45:07.978214 17337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "5dc99fc597ad4b9cb8736172eb5b7b01"
format_stamp: "Formatted at 2026-08-10 06:45:07 on dist-test-slave-vxj2"
server_key: "69b0137cab75cbd76005c364c3476b9f"
server_key_iv: "23eb742ffdd461b8003d39539550314e"
server_key_version: "encryptionkey@0"
I20260810 06:45:07.979662 17337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:07.980408 17355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:07.980589 17337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:07.980700 17337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "5dc99fc597ad4b9cb8736172eb5b7b01"
format_stamp: "Formatted at 2026-08-10 06:45:07 on dist-test-slave-vxj2"
server_key: "69b0137cab75cbd76005c364c3476b9f"
server_key_iv: "23eb742ffdd461b8003d39539550314e"
server_key_version: "encryptionkey@0"
I20260810 06:45:07.980775 17337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:08.018103 17337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:08.020236 17337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:08.020417 17337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:08.020674 17337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:08.021008 17337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:08.021062 17337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:08.021109 17337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:08.021130 17337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:08.027310 17337 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:36715
I20260810 06:45:08.027365 17484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:36715 every 8 connection(s)
I20260810 06:45:08.027741 17337 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:08.033694 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 17337
I20260810 06:45:08.033793 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:08.034019 16291 external_mini_cluster.cc:1428] Setting key 439a3956815fe1fd4a2fe94ee96d41b5
Aug 10 06:45:08 dist-test-slave-vxj2 krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344307, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:08.037340 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.028538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:47357 (local address 127.15.232.254:43437)
0810 06:45:08.028621 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:08.028624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:08.028812 (+ 188us) server_negotiation.cc:407] Connection header received
0810 06:45:08.029892 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:08.029893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:08.029905 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:08.029965 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:08.030567 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.031059 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.032034 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.032222 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.032277 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.034857 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:08.034859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:08.034859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:08.034867 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:08.036119 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:08.036508 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:08.036508 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:08.036509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:08.036560 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:08.036774 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:08.036775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:08.036775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:08.036873 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:08.036923 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:08.037148 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 06:45:08.037276 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:45:08.037626 17487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.028505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:43437 (local address 127.15.232.195:47357)
0810 06:45:08.028684 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:08.028721 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:08.029775 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:08.030024 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:08.030027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:08.030292 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:08.030492 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:08.030497 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.031145 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.031146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:08.031956 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:08.031958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.032298 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.032299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:08.032412 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.032913 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:08.032921 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:08.034735 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:08.036188 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:08.036194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:08.036196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:08.036441 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:08.036617 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:08.036617 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:08.036617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:08.036717 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:08.036931 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:08.036936 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:08.037052 (+ 116us) client_negotiation.cc:769] Sending connection context
0810 06:45:08.037137 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:45:08.037341 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":111,"threads_started":1}
I20260810 06:45:08.038081 17485 heartbeater.cc:347] Connected to a master server at 127.15.232.254:43437
I20260810 06:45:08.038192 17485 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:08.038373 17485 heartbeater.cc:511] Master 127.15.232.254:43437 requested a full tablet report, sending...
I20260810 06:45:08.038731 16962 ts_manager.cc:194] Registered new tserver with Master: 5dc99fc597ad4b9cb8736172eb5b7b01 (127.15.232.195:36715)
I20260810 06:45:08.039083 16962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:47357
I20260810 06:45:08.046065 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:45:08.054064 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.047961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60522 (local address 127.15.232.254:43437)
0810 06:45:08.048028 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:08.048030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:08.048131 (+ 101us) server_negotiation.cc:407] Connection header received
0810 06:45:08.048180 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:08.048181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:08.048191 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:08.048252 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:08.048957 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.049533 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.050327 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.050527 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.050587 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.051757 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:08.051758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:08.051758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:08.051764 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:08.053047 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:08.053392 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:08.053393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:08.053393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:08.053443 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:08.053610 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:08.053610 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:08.053610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:08.053746 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:08.053823 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:08.053933 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 06:45:08.053996 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:45:08.056308 16962 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:08.058153 16962 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:08.064699 17496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.060932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:36715 (local address 127.0.0.1:58270)
0810 06:45:08.061337 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:08.061345 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:08.061388 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:08.061874 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:08.061875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:08.061877 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:08.061924 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:08.061926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.062808 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.062809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:08.063775 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:08.063778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.064470 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.064470 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:08.064536 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.064548 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 06:45:08.064598 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 06:45:08.064639 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":80,"threads_started":1}
I20260810 06:45:08.064752 17503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.061078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58270 (local address 127.15.232.195:36715)
0810 06:45:08.061628 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:08.061632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:08.061639 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:45:08.061669 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:08.061686 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:08.061703 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:08.061791 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:08.062002 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.062688 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.063845 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.064385 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.064473 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.064543 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:08.064623 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 06:45:08.064667 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":138,"threads_started":1}
I20260810 06:45:08.065605 17501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.060681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:43531 (local address 127.0.0.1:52260)
0810 06:45:08.061202 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:08.061211 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:08.061253 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:08.061481 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:08.061485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:08.061486 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:08.061721 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:08.061723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.063076 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.063077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:08.064318 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:08.064321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.065335 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.065336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:08.065406 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.065410 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:08.065474 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 06:45:08.065519 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":54,"thread_start_us":75,"threads_started":1}
I20260810 06:45:08.065614 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.060841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52260 (local address 127.15.232.193:43531)
0810 06:45:08.061325 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:08.061329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:08.061340 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:08.061378 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:08.061380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:08.061395 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:08.061477 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:08.061817 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.062941 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.064428 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.065252 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.065313 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.065346 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:08.065500 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 06:45:08.065538 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":89,"threads_started":1}
I20260810 06:45:08.065564 17409 tablet_service.cc:1467] Processing CreateTablet for tablet 4ef323ac4f0c450fa098e90a504f13d9 (DEFAULT_TABLE table=test-table [id=ece37e4d0a9c4c388b46b8f742d84b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:08.066100 17409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ef323ac4f0c450fa098e90a504f13d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:08.066645 17502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.060978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55270 (local address 127.15.232.194:34109)
0810 06:45:08.061200 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:08.061206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:08.061218 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:45:08.061353 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:08.061356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:08.061374 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:08.061469 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:08.061806 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.062957 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.065378 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.066248 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.066333 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.066386 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:08.066527 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 06:45:08.066568 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":114,"threads_started":1}
I20260810 06:45:08.066642 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.060681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:34109 (local address 127.0.0.1:55270)
0810 06:45:08.061084 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:08.061134 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:08.061234 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:08.061481 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:08.061484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:08.061486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:08.061721 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:08.061723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.063078 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.063078 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:08.064318 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:08.064321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.066330 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.066331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:08.066396 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.066400 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:08.066502 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 06:45:08.066549 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":62,"thread_start_us":63,"threads_started":1}
I20260810 06:45:08.066635 17106 tablet_service.cc:1467] Processing CreateTablet for tablet 4ef323ac4f0c450fa098e90a504f13d9 (DEFAULT_TABLE table=test-table [id=ece37e4d0a9c4c388b46b8f742d84b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:08.066982 17106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ef323ac4f0c450fa098e90a504f13d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:08.069644 17259 tablet_service.cc:1467] Processing CreateTablet for tablet 4ef323ac4f0c450fa098e90a504f13d9 (DEFAULT_TABLE table=test-table [id=ece37e4d0a9c4c388b46b8f742d84b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:08.069859 17504 tablet_bootstrap.cc:492] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01: Bootstrap starting.
I20260810 06:45:08.070075 17259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ef323ac4f0c450fa098e90a504f13d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:08.070475 17505 tablet_bootstrap.cc:492] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5: Bootstrap starting.
I20260810 06:45:08.070668 17504 tablet_bootstrap.cc:654] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:08.071162 17505 tablet_bootstrap.cc:654] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:08.071328 17504 log.cc:824] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:08.071803 17505 log.cc:824] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:08.072201 17504 tablet_bootstrap.cc:492] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01: No bootstrap required, opened a new log
I20260810 06:45:08.072279 17504 ts_tablet_manager.cc:1397] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:45:08.072585 17505 tablet_bootstrap.cc:492] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5: No bootstrap required, opened a new log
I20260810 06:45:08.072669 17505 ts_tablet_manager.cc:1397] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:45:08.074201 17504 raft_consensus.cc:348] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 43531 } } peers { permanent_uuid: "93f3c9e0a5644000ab45893afeb7851a" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 34109 } } peers { permanent_uuid: "5dc99fc597ad4b9cb8736172eb5b7b01" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36715 } }
I20260810 06:45:08.074349 17504 raft_consensus.cc:374] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:08.074373 17504 raft_consensus.cc:729] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc99fc597ad4b9cb8736172eb5b7b01, State: Initialized, Role: FOLLOWER
I20260810 06:45:08.074506 17504 consensus_queue.cc:260] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 43531 } } peers { permanent_uuid: "93f3c9e0a5644000ab45893afeb7851a" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 34109 } } peers { permanent_uuid: "5dc99fc597ad4b9cb8736172eb5b7b01" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36715 } }
I20260810 06:45:08.074445 17505 raft_consensus.cc:348] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 43531 } } peers { permanent_uuid: "93f3c9e0a5644000ab45893afeb7851a" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 34109 } } peers { permanent_uuid: "5dc99fc597ad4b9cb8736172eb5b7b01" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36715 } }
I20260810 06:45:08.074668 17505 raft_consensus.cc:374] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:08.074717 17505 raft_consensus.cc:729] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1ab0f665b594b0fb3cd1df8f0dc34b5, State: Initialized, Role: FOLLOWER
I20260810 06:45:08.074831 17504 ts_tablet_manager.cc:1428] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:08.074868 17485 heartbeater.cc:503] Master 127.15.232.254:43437 was elected leader, sending a full tablet report...
I20260810 06:45:08.074869 17505 consensus_queue.cc:260] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 43531 } } peers { permanent_uuid: "93f3c9e0a5644000ab45893afeb7851a" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 34109 } } peers { permanent_uuid: "5dc99fc597ad4b9cb8736172eb5b7b01" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36715 } }
I20260810 06:45:08.075037 17508 tablet_bootstrap.cc:492] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a: Bootstrap starting.
I20260810 06:45:08.075169 17505 ts_tablet_manager.cc:1428] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:45:08.075275 17182 heartbeater.cc:503] Master 127.15.232.254:43437 was elected leader, sending a full tablet report...
I20260810 06:45:08.075745 17508 tablet_bootstrap.cc:654] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:08.076372 17508 log.cc:824] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:08.077105 17508 tablet_bootstrap.cc:492] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a: No bootstrap required, opened a new log
I20260810 06:45:08.077234 17508 ts_tablet_manager.cc:1397] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:45:08.078899 17508 raft_consensus.cc:348] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 43531 } } peers { permanent_uuid: "93f3c9e0a5644000ab45893afeb7851a" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 34109 } } peers { permanent_uuid: "5dc99fc597ad4b9cb8736172eb5b7b01" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36715 } }
I20260810 06:45:08.079041 17508 raft_consensus.cc:374] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:08.079062 17508 raft_consensus.cc:729] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f3c9e0a5644000ab45893afeb7851a, State: Initialized, Role: FOLLOWER
I20260810 06:45:08.079173 17508 consensus_queue.cc:260] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 43531 } } peers { permanent_uuid: "93f3c9e0a5644000ab45893afeb7851a" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 34109 } } peers { permanent_uuid: "5dc99fc597ad4b9cb8736172eb5b7b01" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36715 } }
I20260810 06:45:08.079443 17508 ts_tablet_manager.cc:1428] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:45:08.079484 17330 heartbeater.cc:503] Master 127.15.232.254:43437 was elected leader, sending a full tablet report...
W20260810 06:45:08.090677 17331 tablet.cc:2369] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:45:08.123667 17510 raft_consensus.cc:482] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:08.123854 17510 raft_consensus.cc:504] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 43531 } } peers { permanent_uuid: "93f3c9e0a5644000ab45893afeb7851a" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 34109 } } peers { permanent_uuid: "5dc99fc597ad4b9cb8736172eb5b7b01" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36715 } }
I20260810 06:45:08.124372 17510 leader_election.cc:288] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93f3c9e0a5644000ab45893afeb7851a (127.15.232.194:34109), 5dc99fc597ad4b9cb8736172eb5b7b01 (127.15.232.195:36715)
I20260810 06:45:08.127381 17503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.124680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:57421 (local address 127.15.232.195:36715)
0810 06:45:08.124753 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:08.124756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:08.124781 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:45:08.124808 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:08.124809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:08.124817 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:08.124899 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:08.125074 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.125629 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.126602 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.126985 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.127085 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.127094 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:08.127265 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 06:45:08.127305 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:45:08.127381 17184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.124575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:36715 (local address 127.15.232.193:57421)
0810 06:45:08.124624 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:08.124636 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:08.124681 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:08.124909 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:08.124910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:08.124912 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:08.124994 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:08.124996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.125719 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.125720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:08.126496 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:08.126499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.127097 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.127098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:08.127184 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.127187 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:08.127266 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:45:08.127305 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 06:45:08.127573 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.124575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:34109 (local address 127.15.232.193:43309)
0810 06:45:08.124948 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:08.124957 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:08.125034 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:08.125224 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:08.125225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:08.125227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:08.125304 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:08.125305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.125925 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.125927 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:08.126696 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:08.126698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.127296 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.127297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:08.127375 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.127379 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:08.127447 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:45:08.127487 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":158,"thread_start_us":150,"threads_started":1}
I20260810 06:45:08.127559 17502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.124689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:43309 (local address 127.15.232.194:34109)
0810 06:45:08.124766 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:08.124769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:08.125027 (+ 258us) server_negotiation.cc:407] Connection header received
0810 06:45:08.125099 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:08.125100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:08.125109 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:08.125212 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:08.125390 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.125842 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.126790 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.127202 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.127294 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.127302 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:08.127453 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 06:45:08.127490 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:45:08.127835 17433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ef323ac4f0c450fa098e90a504f13d9" candidate_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc99fc597ad4b9cb8736172eb5b7b01" is_pre_election: true
I20260810 06:45:08.128059 17433 raft_consensus.cc:2454] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1ab0f665b594b0fb3cd1df8f0dc34b5 in term 0.
I20260810 06:45:08.128294 17281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ef323ac4f0c450fa098e90a504f13d9" candidate_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f3c9e0a5644000ab45893afeb7851a" is_pre_election: true
I20260810 06:45:08.128369 17058 leader_election.cc:302] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dc99fc597ad4b9cb8736172eb5b7b01, c1ab0f665b594b0fb3cd1df8f0dc34b5; no voters:
I20260810 06:45:08.128429 17281 raft_consensus.cc:2454] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1ab0f665b594b0fb3cd1df8f0dc34b5 in term 0.
I20260810 06:45:08.128535 17510 raft_consensus.cc:2781] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:08.128605 17510 raft_consensus.cc:482] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:08.128636 17510 raft_consensus.cc:3037] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:08.129560 17510 raft_consensus.cc:504] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 43531 } } peers { permanent_uuid: "93f3c9e0a5644000ab45893afeb7851a" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 34109 } } peers { permanent_uuid: "5dc99fc597ad4b9cb8736172eb5b7b01" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36715 } }
I20260810 06:45:08.129714 17510 leader_election.cc:288] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [CANDIDATE]: Term 1 election: Requested vote from peers 93f3c9e0a5644000ab45893afeb7851a (127.15.232.194:34109), 5dc99fc597ad4b9cb8736172eb5b7b01 (127.15.232.195:36715)
I20260810 06:45:08.129925 17433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ef323ac4f0c450fa098e90a504f13d9" candidate_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc99fc597ad4b9cb8736172eb5b7b01"
I20260810 06:45:08.129985 17281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ef323ac4f0c450fa098e90a504f13d9" candidate_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f3c9e0a5644000ab45893afeb7851a"
I20260810 06:45:08.130025 17433 raft_consensus.cc:3037] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:08.130053 17281 raft_consensus.cc:3037] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:08.130975 17433 raft_consensus.cc:2454] T 4ef323ac4f0c450fa098e90a504f13d9 P 5dc99fc597ad4b9cb8736172eb5b7b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1ab0f665b594b0fb3cd1df8f0dc34b5 in term 1.
I20260810 06:45:08.130975 17281 raft_consensus.cc:2454] T 4ef323ac4f0c450fa098e90a504f13d9 P 93f3c9e0a5644000ab45893afeb7851a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1ab0f665b594b0fb3cd1df8f0dc34b5 in term 1.
I20260810 06:45:08.131187 17058 leader_election.cc:302] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dc99fc597ad4b9cb8736172eb5b7b01, c1ab0f665b594b0fb3cd1df8f0dc34b5; no voters:
I20260810 06:45:08.131309 17510 raft_consensus.cc:2781] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:08.131536 17510 raft_consensus.cc:686] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [term 1 LEADER]: Becoming Leader. State: Replica: c1ab0f665b594b0fb3cd1df8f0dc34b5, State: Running, Role: LEADER
I20260810 06:45:08.131661 17510 consensus_queue.cc:237] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 43531 } } peers { permanent_uuid: "93f3c9e0a5644000ab45893afeb7851a" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 34109 } } peers { permanent_uuid: "5dc99fc597ad4b9cb8736172eb5b7b01" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36715 } }
I20260810 06:45:08.132640 16962 catalog_manager.cc:5495] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1ab0f665b594b0fb3cd1df8f0dc34b5 (127.15.232.193). New cstate: current_term: 1 leader_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1ab0f665b594b0fb3cd1df8f0dc34b5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 43531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93f3c9e0a5644000ab45893afeb7851a" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 34109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dc99fc597ad4b9cb8736172eb5b7b01" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36715 } health_report { overall_health: UNKNOWN } } }
Aug 10 06:45:08 dist-test-slave-vxj2 krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.193@KRBTEST.COM
I20260810 06:45:08.144233 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.136205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52274 (local address 127.15.232.193:43531)
0810 06:45:08.136249 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:08.136252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:08.136406 (+ 154us) server_negotiation.cc:407] Connection header received
0810 06:45:08.136445 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:08.136445 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:08.136454 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:08.136500 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:08.137000 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.137609 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.138295 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.138479 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.138530 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.140834 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:08.140843 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:08.140844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:08.140851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:08.142920 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:08.143258 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:08.143261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:08.143262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:08.143327 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:08.143508 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:08.143508 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:08.143509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:08.143664 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:08.143898 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:08.143963 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 06:45:08.144104 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 10 06:45:08 dist-test-slave-vxj2 krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344308, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:45:08 dist-test-slave-vxj2 krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344308, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.232.193@KRBTEST.COM
I20260810 06:45:08.170528 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.162113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52290 (local address 127.15.232.193:43531)
0810 06:45:08.162198 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:08.162201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:08.162307 (+ 106us) server_negotiation.cc:407] Connection header received
0810 06:45:08.162430 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:08.162431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:08.162448 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:08.162522 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:08.163186 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.163692 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.164445 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.164656 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.164737 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.167425 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:08.167427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:08.167429 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:08.167442 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:08.169031 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:08.169531 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:08.169533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:08.169533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:08.169601 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:08.169844 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:08.169845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:08.169845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:08.169968 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:08.170090 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:08.170310 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 06:45:08.170410 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260810 06:45:08.175117 17183 tablet.cc:2369] T 4ef323ac4f0c450fa098e90a504f13d9 P c1ab0f665b594b0fb3cd1df8f0dc34b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:45:08 dist-test-slave-vxj2 krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:45:08 dist-test-slave-vxj2 krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.193@KRBTEST.COM
I20260810 06:45:08.194926 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:08.187410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52302 (local address 127.15.232.193:43531)
0810 06:45:08.187489 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:08.187491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:08.187579 (+ 88us) server_negotiation.cc:407] Connection header received
0810 06:45:08.187680 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:08.187681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:08.187692 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:08.187771 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:08.188357 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.188841 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.189666 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:08.189869 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:08.189958 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:08.192292 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:08.192293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:08.192293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:08.192300 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:08.193947 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:08.194261 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:08.194262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:08.194262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:08.194314 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:08.194482 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:08.194483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:08.194483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:08.194577 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:08.194656 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:08.194793 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 06:45:08.194856 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 06:45:08.195567 17084 tablet_service.cc:2908] Rejecting scan request for tablet 4ef323ac4f0c450fa098e90a504f13d9: Uninitialized: clean time has not yet been initialized
I20260810 06:45:08.196591 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 17035
I20260810 06:45:08.204641 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 17187
I20260810 06:45:08.215114 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 17337
I20260810 06:45:08.223994 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 16928
2026-08-10T06:45:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2168 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:08 dist-test-slave-vxj2 krb5kdc[17529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:08 dist-test-slave-vxj2 krb5kdc[17529](info): set up 2 sockets
Aug 10 06:45:08 dist-test-slave-vxj2 krb5kdc[17529](info): commencing operation
krb5kdc: starting...
W20260810 06:45:09.345892 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:09 dist-test-slave-vxj2 krb5kdc[17529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:45:09.480334 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:34133
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:34133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:09.566633 17545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:09.566879 17545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:09.566918 17545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:09.568086 17545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:09.568152 17545 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:45:09.568176 17545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:09.568210 17545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:09.568238 17545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:09.568267 17545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:09.569819 17545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:34133
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:34133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:09.570104 17545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:09.570341 17545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:09.575369 17551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:09.575374 17552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:09.575356 17554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:09.575593 17545 server_base.cc:1034] running on GCE node
I20260810 06:45:09.575958 17545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:09.576213 17545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:09.577360 17545 hybrid_clock.cc:648] HybridClock initialized: now 1786344309577326 us; error 44 us; skew 500 ppm
Aug 10 06:45:09 dist-test-slave-vxj2 krb5kdc[17529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344309, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:09.586087 17545 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:45:09.586629 17545 webserver.cc:460] Webserver started at http://127.15.232.254:43293/ using document root <none> and password file <none>
I20260810 06:45:09.586884 17545 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:09.586956 17545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:09.587116 17545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:09.588255 17545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "36fd495f17b240aeb3d629b908dc6a54"
format_stamp: "Formatted at 2026-08-10 06:45:09 on dist-test-slave-vxj2"
server_key: "80783df871570dfbcd0c5b1a8ec7eec8"
server_key_iv: "903a05d98c1c1009b77e708bf0ca4c07"
server_key_version: "encryptionkey@0"
I20260810 06:45:09.588665 17545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "36fd495f17b240aeb3d629b908dc6a54"
format_stamp: "Formatted at 2026-08-10 06:45:09 on dist-test-slave-vxj2"
server_key: "80783df871570dfbcd0c5b1a8ec7eec8"
server_key_iv: "903a05d98c1c1009b77e708bf0ca4c07"
server_key_version: "encryptionkey@0"
I20260810 06:45:09.590197 17545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:09.591042 17561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:09.591246 17545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:09.591382 17545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "36fd495f17b240aeb3d629b908dc6a54"
format_stamp: "Formatted at 2026-08-10 06:45:09 on dist-test-slave-vxj2"
server_key: "80783df871570dfbcd0c5b1a8ec7eec8"
server_key_iv: "903a05d98c1c1009b77e708bf0ca4c07"
server_key_version: "encryptionkey@0"
I20260810 06:45:09.591462 17545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:09.605345 17545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:09.607717 17545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:09.607906 17545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:09.613786 17545 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:34133
I20260810 06:45:09.613842 17628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:34133 every 8 connection(s)
I20260810 06:45:09.614212 17545 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:09.614912 17629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:09.616143 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 17545
I20260810 06:45:09.616246 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:09.616413 16291 external_mini_cluster.cc:1428] Setting key aa5217d25b7d27d1e7267130a4edc4e2
I20260810 06:45:09.617784 17629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54: Bootstrap starting.
I20260810 06:45:09.618568 17629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:09.618932 17629 log.cc:824] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:09.619665 17629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54: No bootstrap required, opened a new log
I20260810 06:45:09.622033 17629 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fd495f17b240aeb3d629b908dc6a54" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34133 } }
I20260810 06:45:09.622187 17629 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:09.622238 17629 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fd495f17b240aeb3d629b908dc6a54, State: Initialized, Role: FOLLOWER
I20260810 06:45:09.622406 17629 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fd495f17b240aeb3d629b908dc6a54" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34133 } }
I20260810 06:45:09.622507 17629 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:09.622546 17629 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:09.622601 17629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:09.623421 17629 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fd495f17b240aeb3d629b908dc6a54" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34133 } }
I20260810 06:45:09.623548 17629 leader_election.cc:302] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36fd495f17b240aeb3d629b908dc6a54; no voters:
I20260810 06:45:09.623745 17629 leader_election.cc:288] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:09.623781 17635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:09.623906 17635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [term 1 LEADER]: Becoming Leader. State: Replica: 36fd495f17b240aeb3d629b908dc6a54, State: Running, Role: LEADER
I20260810 06:45:09.623991 17635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fd495f17b240aeb3d629b908dc6a54" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34133 } }
I20260810 06:45:09.624028 17629 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [sys.catalog]: configured and running, proceeding with master startup.
Aug 10 06:45:09 dist-test-slave-vxj2 krb5kdc[17529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:09.624403 17637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36fd495f17b240aeb3d629b908dc6a54. Latest consensus state: current_term: 1 leader_uuid: "36fd495f17b240aeb3d629b908dc6a54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fd495f17b240aeb3d629b908dc6a54" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34133 } } }
I20260810 06:45:09.624430 17636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36fd495f17b240aeb3d629b908dc6a54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fd495f17b240aeb3d629b908dc6a54" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34133 } } }
I20260810 06:45:09.624537 17637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:09.624558 17636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:09.624871 17644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:09.625478 17644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:09.627393 17644 catalog_manager.cc:1347] Generated new cluster ID: 574c468f5a3d480c9d5a58b2deb50a6d
I20260810 06:45:09.627446 17644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:09.629223 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:09.616971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51856 (local address 127.15.232.254:34133)
0810 06:45:09.617134 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:09.617143 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:09.617194 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:45:09.617616 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:09.617622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:09.617669 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:09.617846 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:09.618571 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:09.620061 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:09.621221 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:09.621661 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:09.621754 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:09.624825 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:09.624830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:09.624830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:09.624844 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:09.627382 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:09.627840 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:09.627844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:09.627844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:09.627962 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:09.628205 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:09.628206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:09.628207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:09.628478 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:09.628629 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:09.628767 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 06:45:09.628923 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260810 06:45:09.662038 17644 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:09.662624 17644 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:09.667737 17644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54: Generated new TSK 0
I20260810 06:45:09.667974 17644 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:45:09.704939 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:34133
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:09.806036 17658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:09.806264 17658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:09.806293 17658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:09.807905 17658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:09.807973 17658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:09.807997 17658 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:45:09.808018 17658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:09.809721 17658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.254:34133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:09.809993 17658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:09.810272 17658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:09.815707 17665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:09.815713 17666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:09.815719 17668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:09.816320 17658 server_base.cc:1034] running on GCE node
I20260810 06:45:09.816540 17658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:09.816800 17658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:09.817957 17658 hybrid_clock.cc:648] HybridClock initialized: now 1786344309817943 us; error 26 us; skew 500 ppm
Aug 10 06:45:09 dist-test-slave-vxj2 krb5kdc[17529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344309, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:09.821293 17658 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:45:09.821846 17658 webserver.cc:460] Webserver started at http://127.15.232.193:45885/ using document root <none> and password file <none>
I20260810 06:45:09.822150 17658 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:09.822230 17658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:09.822402 17658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:09.823588 17658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "c5da9a732f9b40758b9f3c9e7501f8e7"
format_stamp: "Formatted at 2026-08-10 06:45:09 on dist-test-slave-vxj2"
server_key: "b77176ac273745cdbab9463279991054"
server_key_iv: "fb498c8aa23f1492b58549cdc079f555"
server_key_version: "encryptionkey@0"
I20260810 06:45:09.824069 17658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "c5da9a732f9b40758b9f3c9e7501f8e7"
format_stamp: "Formatted at 2026-08-10 06:45:09 on dist-test-slave-vxj2"
server_key: "b77176ac273745cdbab9463279991054"
server_key_iv: "fb498c8aa23f1492b58549cdc079f555"
server_key_version: "encryptionkey@0"
I20260810 06:45:09.825767 17658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:09.826716 17679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:09.826942 17658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:09.827069 17658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "c5da9a732f9b40758b9f3c9e7501f8e7"
format_stamp: "Formatted at 2026-08-10 06:45:09 on dist-test-slave-vxj2"
server_key: "b77176ac273745cdbab9463279991054"
server_key_iv: "fb498c8aa23f1492b58549cdc079f555"
server_key_version: "encryptionkey@0"
I20260810 06:45:09.827165 17658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:09.837595 17658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:09.840175 17658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:09.840363 17658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:09.840922 17658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:09.840988 17658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:09.841064 17658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:09.841100 17658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:45:09 dist-test-slave-vxj2 krb5kdc[17529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344309, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:09.849313 17658 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:37149
I20260810 06:45:09.849404 17814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:37149 every 8 connection(s)
I20260810 06:45:09.849789 17658 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:09.851207 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 17658
I20260810 06:45:09.851338 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:09.851544 16291 external_mini_cluster.cc:1428] Setting key 9d5b5c860d1d6fe790936c1853b33a7e
I20260810 06:45:09.853119 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:09.841539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:35847 (local address 127.15.232.254:34133)
0810 06:45:09.841614 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:09.841617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:09.841864 (+ 247us) server_negotiation.cc:407] Connection header received
0810 06:45:09.842359 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:09.842361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:09.842377 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:09.842468 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:09.843382 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:09.844332 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:09.845787 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:09.846135 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:09.846222 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:09.849509 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:09.849511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:09.849511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:09.849521 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:09.851223 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:09.851885 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:09.851903 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:09.851904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:09.851981 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:09.852306 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:09.852307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:09.852307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:09.852476 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:09.852539 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:09.852966 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 06:45:09.853048 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:45:09.853421 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:09.841513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34133 (local address 127.15.232.193:35847)
0810 06:45:09.841709 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:09.841750 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:09.842246 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:09.842536 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:09.842539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:09.842914 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:09.843255 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:09.843263 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:09.844454 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:09.844456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:09.845692 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:09.845695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:09.846276 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:09.846278 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:09.846507 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:09.847132 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:09.847143 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:09.849404 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:09.851351 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:09.851359 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:09.851362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:09.851728 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:09.852074 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:09.852075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:09.852075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:09.852233 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:09.852553 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:09.852560 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:09.852791 (+ 231us) client_negotiation.cc:769] Sending connection context
0810 06:45:09.852935 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 06:45:09.853129 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":129,"threads_started":1}
I20260810 06:45:09.854322 17815 heartbeater.cc:347] Connected to a master server at 127.15.232.254:34133
I20260810 06:45:09.854413 17815 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:09.854614 17815 heartbeater.cc:511] Master 127.15.232.254:34133 requested a full tablet report, sending...
I20260810 06:45:09.855435 17582 ts_manager.cc:194] Registered new tserver with Master: c5da9a732f9b40758b9f3c9e7501f8e7 (127.15.232.193:37149)
I20260810 06:45:09.856081 17582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:35847
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
I20260810 06:45:09.862874 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:09.856496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:56965 (local address 127.15.232.254:34133)
0810 06:45:09.857038 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:09.857040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:09.857048 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:09.857065 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:09.857065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:09.857073 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:09.857149 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:09.857553 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:09.858023 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:09.858842 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:09.859039 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:09.859125 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:09.860091 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:09.860092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:09.860092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:09.860100 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:09.861641 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:09.861998 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:09.861999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:09.862000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:09.862060 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:09.862280 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:09.862281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:09.862281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:09.862396 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:09.862475 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:09.862674 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 06:45:09.862767 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521}
I20260810 06:45:09.862885 17822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:09.856440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34133 (local address 127.15.232.193:56965)
0810 06:45:09.856586 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:09.856599 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:09.856650 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:09.857209 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:09.857210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:09.857394 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:09.857463 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:09.857464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:09.858119 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:09.858120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:09.858758 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:09.858760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:09.859127 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:09.859128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:09.859205 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:09.859683 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:09.859688 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:09.859995 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:09.861733 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:09.861734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:09.861735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:09.861919 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:09.862129 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:09.862130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:09.862131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:09.862198 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:09.862477 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:09.862478 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:09.862564 (+ 86us) client_negotiation.cc:769] Sending connection context
0810 06:45:09.862633 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 06:45:09.862781 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:45:09.881506 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:34133
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:09.966027 17824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:09.966272 17824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:09.966317 17824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:09.967612 17824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:09.967686 17824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:09.967722 17824 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:45:09.967772 17824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:09.969456 17824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.254:34133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:09.969744 17824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:09.970009 17824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:09.975441 17829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:09.975435 17830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:09.975531 17824 server_base.cc:1034] running on GCE node
W20260810 06:45:09.975431 17832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:09.975896 17824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:09.976169 17824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:09.977347 17824 hybrid_clock.cc:648] HybridClock initialized: now 1786344309977346 us; error 37 us; skew 500 ppm
Aug 10 06:45:09 dist-test-slave-vxj2 krb5kdc[17529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344309, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:09.980123 17824 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:45:09.980599 17824 webserver.cc:460] Webserver started at http://127.15.232.194:39543/ using document root <none> and password file <none>
I20260810 06:45:09.980901 17824 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:09.980978 17824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:09.981128 17824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:09.982272 17824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "1fa096ac601144d6877084cb980cc4ca"
format_stamp: "Formatted at 2026-08-10 06:45:09 on dist-test-slave-vxj2"
server_key: "f44d2b0b7c14a23d4a630d69103cbe31"
server_key_iv: "1dcc63514f6526195747b388292b5bae"
server_key_version: "encryptionkey@0"
I20260810 06:45:09.982649 17824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "1fa096ac601144d6877084cb980cc4ca"
format_stamp: "Formatted at 2026-08-10 06:45:09 on dist-test-slave-vxj2"
server_key: "f44d2b0b7c14a23d4a630d69103cbe31"
server_key_iv: "1dcc63514f6526195747b388292b5bae"
server_key_version: "encryptionkey@0"
I20260810 06:45:09.984078 17824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:09.984866 17842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:09.985100 17824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:09.985248 17824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "1fa096ac601144d6877084cb980cc4ca"
format_stamp: "Formatted at 2026-08-10 06:45:09 on dist-test-slave-vxj2"
server_key: "f44d2b0b7c14a23d4a630d69103cbe31"
server_key_iv: "1dcc63514f6526195747b388292b5bae"
server_key_version: "encryptionkey@0"
I20260810 06:45:09.985335 17824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:10.007676 17824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:10.009891 17824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:10.010082 17824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:10.010713 17824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:10.010788 17824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:10.010829 17824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:10.010854 17824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:45:10 dist-test-slave-vxj2 krb5kdc[17529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344309, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:10.018723 17824 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:33397
I20260810 06:45:10.018805 17968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:33397 every 8 connection(s)
I20260810 06:45:10.019131 17824 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:10.021984 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.011306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:34733 (local address 127.15.232.254:34133)
0810 06:45:10.011392 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.011396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.011589 (+ 193us) server_negotiation.cc:407] Connection header received
0810 06:45:10.012099 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.012101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.012115 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.012188 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:10.013119 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.013676 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.014740 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.014947 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.015030 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.018925 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:10.018927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:10.018928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:10.018936 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:10.020574 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.021018 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.021019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.021019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.021078 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.021399 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.021400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.021400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.021521 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:10.021579 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.021836 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.021916 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:45:10.022215 17851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.011284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34133 (local address 127.15.232.194:34733)
0810 06:45:10.011446 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.011490 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.011977 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.012276 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.012280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.012675 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:10.013025 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.013033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.013785 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.013786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.014631 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.014634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.015038 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.015040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.015188 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.015763 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:10.015777 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:10.018792 (+ 3015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:10.020670 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:10.020676 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:10.020678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:10.020934 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:10.021142 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:10.021143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:10.021143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:10.021321 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:10.021586 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:10.021591 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:10.021716 (+ 125us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.021802 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.021980 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260810 06:45:10.022782 17969 heartbeater.cc:347] Connected to a master server at 127.15.232.254:34133
I20260810 06:45:10.022915 17969 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:10.023124 17969 heartbeater.cc:511] Master 127.15.232.254:34133 requested a full tablet report, sending...
I20260810 06:45:10.023566 17583 ts_manager.cc:194] Registered new tserver with Master: 1fa096ac601144d6877084cb980cc4ca (127.15.232.194:33397)
I20260810 06:45:10.024111 17583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:34733
I20260810 06:45:10.027244 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 17824
I20260810 06:45:10.027362 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:10.027565 16291 external_mini_cluster.cc:1428] Setting key de670121563e8817604927433a16941b
I20260810 06:45:10.031505 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.024102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:54153 (local address 127.15.232.254:34133)
0810 06:45:10.024145 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.024147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.024221 (+ 74us) server_negotiation.cc:407] Connection header received
0810 06:45:10.024303 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.024304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.024315 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.024369 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:10.024823 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.025369 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.026684 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.026926 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.026990 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.028181 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:10.028182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:10.028182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:10.028189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:10.030079 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.030658 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.030660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.030660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.030728 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.030891 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.030892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.030892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.031069 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:10.031145 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.031279 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.031383 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:45:10.031558 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.024038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34133 (local address 127.15.232.194:54153)
0810 06:45:10.024164 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.024175 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.024234 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.024380 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.024380 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.024672 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:10.024742 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.024745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.025472 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.025473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.026569 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.026573 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.027012 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.027013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.027092 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.027644 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:10.027650 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:10.028071 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:10.030357 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:10.030358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:10.030358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:10.030584 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:10.030803 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:10.030803 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:10.030803 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:10.030852 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:10.031122 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:10.031124 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:10.031179 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.031254 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.031398 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:45:10.056780 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:34133
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:10.141461 17978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:10.141674 17978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:10.141700 17978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:10.142879 17978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:10.142922 17978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:10.142943 17978 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:45:10.142989 17978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:10.144438 17978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.254:34133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:10.144688 17978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:10.144925 17978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:10.150046 17985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:10.150065 17987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:10.150089 17989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:10.150358 17978 server_base.cc:1034] running on GCE node
I20260810 06:45:10.150657 17978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:10.150915 17978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:10.152087 17978 hybrid_clock.cc:648] HybridClock initialized: now 1786344310152074 us; error 26 us; skew 500 ppm
Aug 10 06:45:10 dist-test-slave-vxj2 krb5kdc[17529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344310, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:10.154878 17978 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:45:10.155323 17978 webserver.cc:460] Webserver started at http://127.15.232.195:44909/ using document root <none> and password file <none>
I20260810 06:45:10.155557 17978 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:10.155624 17978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:10.155766 17978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:10.156844 17978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8"
format_stamp: "Formatted at 2026-08-10 06:45:10 on dist-test-slave-vxj2"
server_key: "ec25f0fff59d5f986b5146c2eaa0124c"
server_key_iv: "db7200d18296e8f929da37e5148861c6"
server_key_version: "encryptionkey@0"
I20260810 06:45:10.157279 17978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8"
format_stamp: "Formatted at 2026-08-10 06:45:10 on dist-test-slave-vxj2"
server_key: "ec25f0fff59d5f986b5146c2eaa0124c"
server_key_iv: "db7200d18296e8f929da37e5148861c6"
server_key_version: "encryptionkey@0"
I20260810 06:45:10.158712 17978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:10.159561 17998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:10.159775 17978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:10.159893 17978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8"
format_stamp: "Formatted at 2026-08-10 06:45:10 on dist-test-slave-vxj2"
server_key: "ec25f0fff59d5f986b5146c2eaa0124c"
server_key_iv: "db7200d18296e8f929da37e5148861c6"
server_key_version: "encryptionkey@0"
I20260810 06:45:10.159983 17978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:10.166553 17978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:10.168655 17978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:10.168813 17978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:10.169400 17978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:10.169461 17978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:10.169507 17978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:10.169533 17978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:10.176512 17978 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:43287
I20260810 06:45:10.176596 18130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:43287 every 8 connection(s)
Aug 10 06:45:10 dist-test-slave-vxj2 krb5kdc[17529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344310, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:10.176962 17978 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:10.179968 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.169834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:58049 (local address 127.15.232.254:34133)
0810 06:45:10.169895 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.169898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.170110 (+ 212us) server_negotiation.cc:407] Connection header received
0810 06:45:10.170581 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.170582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.170596 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.170679 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:10.171578 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.172442 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.174117 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.174465 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.174558 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.177423 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:10.177424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:10.177425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:10.177433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:10.178726 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.179155 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.179155 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.179156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.179209 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.179426 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.179426 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.179427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.179528 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:10.179580 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.179810 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.179905 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 06:45:10.180162 18010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.169822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34133 (local address 127.15.232.195:58049)
0810 06:45:10.169975 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.170015 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.170488 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.170769 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.170774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.171155 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:10.171445 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.171453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.172544 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.172545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.174020 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.174023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.174587 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.174588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.174727 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.175310 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:10.175329 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:10.177331 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:10.178839 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:10.178846 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:10.178847 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:10.179082 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:10.179269 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:10.179270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:10.179270 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:10.179370 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:10.179584 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:10.179588 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:10.179698 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.179781 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.179955 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260810 06:45:10.180771 18131 heartbeater.cc:347] Connected to a master server at 127.15.232.254:34133
I20260810 06:45:10.180867 18131 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:10.181066 18131 heartbeater.cc:511] Master 127.15.232.254:34133 requested a full tablet report, sending...
I20260810 06:45:10.181514 17583 ts_manager.cc:194] Registered new tserver with Master: 2eaa13ea68cd43a2b3d27ffe950ab2f8 (127.15.232.195:43287)
I20260810 06:45:10.181998 17583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:58049
I20260810 06:45:10.182216 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 17978
I20260810 06:45:10.182302 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:10.182462 16291 external_mini_cluster.cc:1428] Setting key c60fdad5dfb775b2417b6ce8c08a3866
I20260810 06:45:10.183238 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:45:10.188508 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.181920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34133 (local address 127.15.232.195:36717)
0810 06:45:10.182061 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.182073 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.182137 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.182296 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.182296 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.182496 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:10.182609 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.182611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.183499 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.183500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.184014 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.184016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.184411 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.184412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.184486 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.184973 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:10.184980 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:10.185473 (+ 493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:10.187376 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:10.187376 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:10.187377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:10.187610 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:10.187832 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:10.187833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:10.187833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:10.187905 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:10.188188 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:10.188189 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:10.188229 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.188287 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.188392 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260810 06:45:10.188577 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.182012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:36717 (local address 127.15.232.254:34133)
0810 06:45:10.182057 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.182060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.182192 (+ 132us) server_negotiation.cc:407] Connection header received
0810 06:45:10.182214 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.182215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.182224 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.182279 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:10.182718 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.183405 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.184116 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.184324 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.184408 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.185565 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:10.185566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:10.185567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:10.185573 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:10.187278 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.187686 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.187688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.187688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.187756 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.187972 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.187973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.187973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.188122 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:10.188204 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.188321 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.188399 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Aug 10 06:45:10 dist-test-slave-vxj2 krb5kdc[17529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344310, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:45:10 dist-test-slave-vxj2 krb5kdc[17529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344310, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:10.207366 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.200196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51862 (local address 127.15.232.254:34133)
0810 06:45:10.200264 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.200266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.200276 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:10.200315 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.200315 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.200324 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.200374 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:10.200931 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.201496 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.202236 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.202432 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.202485 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.204648 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:10.204649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:10.204650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:10.204657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:10.206212 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.206567 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.206568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.206568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.206640 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.206846 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.206847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.206847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.206969 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:10.207057 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.207154 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.207215 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:45:10.209403 17583 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:10.211014 17583 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:10.217934 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.213574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:37149 (local address 127.0.0.1:56396)
0810 06:45:10.214044 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.214087 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.214182 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.214627 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.214628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.214630 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:10.214664 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.214665 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.215511 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.215513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.216697 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.216700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.217586 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.217587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.217674 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.217688 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.217781 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.217835 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":63,"threads_started":1}
I20260810 06:45:10.218114 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.213574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:43287 (local address 127.0.0.1:33150)
0810 06:45:10.214044 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.214088 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.214188 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.214451 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.214454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.214457 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:10.214535 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.214537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.215916 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.215918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.216834 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.216836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.217779 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.217780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.217845 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.217849 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.217901 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.217933 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":22,"thread_start_us":84,"threads_started":1}
I20260810 06:45:10.218168 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.213763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.15.232.195:43287)
0810 06:45:10.213973 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.213978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.214157 (+ 179us) server_negotiation.cc:407] Connection header received
0810 06:45:10.214318 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.214321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.214339 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.214617 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:10.214702 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.215800 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.216927 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.217693 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.217774 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.217827 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.218003 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.218054 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":89,"threads_started":1}
I20260810 06:45:10.218230 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.213616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33397 (local address 127.0.0.1:60130)
0810 06:45:10.214291 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.214300 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.214348 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.214544 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.214545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.214547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:10.214604 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.214605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.215902 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.215903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.217013 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.217016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.217902 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.217903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.217968 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.217972 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.218119 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.218168 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":21,"thread_start_us":95,"threads_started":1}
I20260810 06:45:10.218499 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.213772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56396 (local address 127.15.232.193:37149)
0810 06:45:10.214047 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.214051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.214165 (+ 114us) server_negotiation.cc:407] Connection header received
0810 06:45:10.214436 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.214440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.214457 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.214557 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:10.214721 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.215421 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.216797 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.217499 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.217589 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.217648 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.218319 (+ 671us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.218381 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":171,"threads_started":1}
I20260810 06:45:10.218918 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.213872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60130 (local address 127.15.232.194:33397)
0810 06:45:10.214382 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.214387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.214397 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:10.214441 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.214443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.214458 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.214538 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:10.214692 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.215808 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.217097 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.217836 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.217902 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.217948 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.218767 (+ 819us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.218820 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":86,"threads_started":1}
I20260810 06:45:10.219128 18052 tablet_service.cc:1467] Processing CreateTablet for tablet 970d204e9a604672af08fab2803aacc6 (DEFAULT_TABLE table=test-table [id=095bf0d58159415584c95de88df238a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:10.219219 17737 tablet_service.cc:1467] Processing CreateTablet for tablet 970d204e9a604672af08fab2803aacc6 (DEFAULT_TABLE table=test-table [id=095bf0d58159415584c95de88df238a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:10.219497 17737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 970d204e9a604672af08fab2803aacc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:10.219553 18052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 970d204e9a604672af08fab2803aacc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:10.219537 17895 tablet_service.cc:1467] Processing CreateTablet for tablet 970d204e9a604672af08fab2803aacc6 (DEFAULT_TABLE table=test-table [id=095bf0d58159415584c95de88df238a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:10.219841 17895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 970d204e9a604672af08fab2803aacc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:10.222833 18157 tablet_bootstrap.cc:492] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7: Bootstrap starting.
I20260810 06:45:10.223402 18159 tablet_bootstrap.cc:492] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca: Bootstrap starting.
I20260810 06:45:10.223619 18158 tablet_bootstrap.cc:492] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Bootstrap starting.
I20260810 06:45:10.223668 18157 tablet_bootstrap.cc:654] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:10.224095 18159 tablet_bootstrap.cc:654] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:10.224177 18157 log.cc:824] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:10.224442 18159 log.cc:824] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:10.224684 18158 tablet_bootstrap.cc:654] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:10.225023 18157 tablet_bootstrap.cc:492] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7: No bootstrap required, opened a new log
I20260810 06:45:10.225049 18158 log.cc:824] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:10.225205 18157 ts_tablet_manager.cc:1397] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:45:10.225522 18159 tablet_bootstrap.cc:492] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca: No bootstrap required, opened a new log
I20260810 06:45:10.225603 18159 ts_tablet_manager.cc:1397] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:45:10.225803 18158 tablet_bootstrap.cc:492] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: No bootstrap required, opened a new log
I20260810 06:45:10.225891 18158 ts_tablet_manager.cc:1397] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 06:45:10.227277 18158 raft_consensus.cc:348] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } }
I20260810 06:45:10.227455 18158 raft_consensus.cc:374] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:10.227325 18157 raft_consensus.cc:348] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } }
I20260810 06:45:10.227504 18158 raft_consensus.cc:729] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eaa13ea68cd43a2b3d27ffe950ab2f8, State: Initialized, Role: FOLLOWER
I20260810 06:45:10.227509 18157 raft_consensus.cc:374] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:10.227543 18157 raft_consensus.cc:729] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5da9a732f9b40758b9f3c9e7501f8e7, State: Initialized, Role: FOLLOWER
I20260810 06:45:10.227643 18158 consensus_queue.cc:260] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } }
I20260810 06:45:10.227666 18157 consensus_queue.cc:260] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } }
I20260810 06:45:10.227610 18159 raft_consensus.cc:348] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } }
I20260810 06:45:10.227738 18159 raft_consensus.cc:374] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:10.227800 18159 raft_consensus.cc:729] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa096ac601144d6877084cb980cc4ca, State: Initialized, Role: FOLLOWER
I20260810 06:45:10.227931 18157 ts_tablet_manager.cc:1428] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:10.227912 18159 consensus_queue.cc:260] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } }
I20260810 06:45:10.227960 18158 ts_tablet_manager.cc:1428] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:45:10.228058 17815 heartbeater.cc:503] Master 127.15.232.254:34133 was elected leader, sending a full tablet report...
I20260810 06:45:10.228099 18131 heartbeater.cc:503] Master 127.15.232.254:34133 was elected leader, sending a full tablet report...
I20260810 06:45:10.228148 18159 ts_tablet_manager.cc:1428] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:10.228256 17969 heartbeater.cc:503] Master 127.15.232.254:34133 was elected leader, sending a full tablet report...
W20260810 06:45:10.269799 17970 tablet.cc:2369] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:45:10.350615 17816 tablet.cc:2369] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:45:10.386605 18165 raft_consensus.cc:482] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:10.386750 18165 raft_consensus.cc:504] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } }
I20260810 06:45:10.387151 18165 leader_election.cc:288] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5da9a732f9b40758b9f3c9e7501f8e7 (127.15.232.193:37149), 2eaa13ea68cd43a2b3d27ffe950ab2f8 (127.15.232.195:43287)
I20260810 06:45:10.390533 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.387720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:49081 (local address 127.15.232.193:37149)
0810 06:45:10.387770 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.387772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.387783 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:10.387810 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.387810 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.387817 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.387870 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:10.388063 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.388591 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.389739 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.390127 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.390184 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.390191 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.390405 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.390450 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:45:10.390534 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.387505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:37149 (local address 127.15.232.194:49081)
0810 06:45:10.387688 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.387698 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.387760 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.387893 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.387895 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.387897 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:10.387989 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.387991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.388687 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.388688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.389630 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.389633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.390208 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.390210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.390328 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.390333 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.390391 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.390430 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":15,"thread_start_us":100,"threads_started":1}
I20260810 06:45:10.390659 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.387653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:41537 (local address 127.15.232.195:43287)
0810 06:45:10.387732 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.387734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.387745 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:10.387845 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.387865 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.387876 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.387935 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:10.388167 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.388803 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.389832 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.390269 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.390342 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.390351 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.390528 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.390579 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:45:10.390883 17851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.387504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:43287 (local address 127.15.232.194:41537)
0810 06:45:10.387586 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.387601 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.387673 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.387988 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.387989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.387992 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:10.388081 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.388083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.388890 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.388891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.389742 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.389745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.390350 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.390351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.390443 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.390447 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.390506 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.390552 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 06:45:10.391013 17759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970d204e9a604672af08fab2803aacc6" candidate_uuid: "1fa096ac601144d6877084cb980cc4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" is_pre_election: true
I20260810 06:45:10.391225 17759 raft_consensus.cc:2454] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fa096ac601144d6877084cb980cc4ca in term 0.
I20260810 06:45:10.391253 18075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970d204e9a604672af08fab2803aacc6" candidate_uuid: "1fa096ac601144d6877084cb980cc4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" is_pre_election: true
I20260810 06:45:10.391431 18075 raft_consensus.cc:2454] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fa096ac601144d6877084cb980cc4ca in term 0.
I20260810 06:45:10.391458 17845 leader_election.cc:302] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa096ac601144d6877084cb980cc4ca, c5da9a732f9b40758b9f3c9e7501f8e7; no voters:
I20260810 06:45:10.391577 18165 raft_consensus.cc:2781] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:10.391655 18165 raft_consensus.cc:482] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:10.391691 18165 raft_consensus.cc:3037] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:10.392642 18165 raft_consensus.cc:504] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } }
I20260810 06:45:10.392807 18165 leader_election.cc:288] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [CANDIDATE]: Term 1 election: Requested vote from peers c5da9a732f9b40758b9f3c9e7501f8e7 (127.15.232.193:37149), 2eaa13ea68cd43a2b3d27ffe950ab2f8 (127.15.232.195:43287)
I20260810 06:45:10.392983 17759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970d204e9a604672af08fab2803aacc6" candidate_uuid: "1fa096ac601144d6877084cb980cc4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7"
I20260810 06:45:10.392987 18075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970d204e9a604672af08fab2803aacc6" candidate_uuid: "1fa096ac601144d6877084cb980cc4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8"
I20260810 06:45:10.393072 18075 raft_consensus.cc:3037] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:10.393077 17759 raft_consensus.cc:3037] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:10.393872 18075 raft_consensus.cc:2454] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fa096ac601144d6877084cb980cc4ca in term 1.
I20260810 06:45:10.393919 17759 raft_consensus.cc:2454] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fa096ac601144d6877084cb980cc4ca in term 1.
I20260810 06:45:10.394100 17843 leader_election.cc:302] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa096ac601144d6877084cb980cc4ca, 2eaa13ea68cd43a2b3d27ffe950ab2f8; no voters:
I20260810 06:45:10.394215 18165 raft_consensus.cc:2781] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:10.394385 18165 raft_consensus.cc:686] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 1 LEADER]: Becoming Leader. State: Replica: 1fa096ac601144d6877084cb980cc4ca, State: Running, Role: LEADER
I20260810 06:45:10.394483 18165 consensus_queue.cc:237] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } }
I20260810 06:45:10.395255 17583 catalog_manager.cc:5495] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fa096ac601144d6877084cb980cc4ca (127.15.232.194). New cstate: current_term: 1 leader_uuid: "1fa096ac601144d6877084cb980cc4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } health_report { overall_health: HEALTHY } } }
W20260810 06:45:10.427662 18133 tablet.cc:2369] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:45:10.446372 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.443446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60132 (local address 127.15.232.194:33397)
0810 06:45:10.443546 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.443548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.443561 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:45:10.443780 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.443782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.443810 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.443903 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:10.444085 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.444630 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.445464 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.445661 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.445743 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.445971 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:10.446078 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:10.446174 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.446225 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.446248 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:45:10.448706 18075 raft_consensus.cc:1261] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 1 FOLLOWER]: Refusing update from remote peer 1fa096ac601144d6877084cb980cc4ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:10.449144 17759 raft_consensus.cc:1261] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 1 FOLLOWER]: Refusing update from remote peer 1fa096ac601144d6877084cb980cc4ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:10.449198 18167 consensus_queue.cc:1035] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:10.449513 18165 consensus_queue.cc:1035] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:10.452911 18171 mvcc.cc:205] Tried to move back new op lower bound from 7316866295593709568 to 7316866295376334848. Current Snapshot: MvccSnapshot[applied={T|T < 7316866295593709568}]
I20260810 06:45:10.453919 18170 mvcc.cc:205] Tried to move back new op lower bound from 7316866295593709568 to 7316866295376334848. Current Snapshot: MvccSnapshot[applied={T|T < 7316866295593709568}]
I20260810 06:45:10.457971 17582 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:10.458109 17582 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:10.459157 17582 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54: Sending DeleteTablet for 3 replicas of tablet 970d204e9a604672af08fab2803aacc6
I20260810 06:45:10.459684 17895 tablet_service.cc:1514] Processing DeleteTablet for tablet 970d204e9a604672af08fab2803aacc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:10 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:60130
I20260810 06:45:10.459707 18052 tablet_service.cc:1514] Processing DeleteTablet for tablet 970d204e9a604672af08fab2803aacc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:10 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:33150
I20260810 06:45:10.460029 17737 tablet_service.cc:1514] Processing DeleteTablet for tablet 970d204e9a604672af08fab2803aacc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:10 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:56396
I20260810 06:45:10.460086 17582 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:10.460243 17582 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:10.460369 18180 tablet_replica.cc:330] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: stopping tablet replica
I20260810 06:45:10.460492 18180 raft_consensus.cc:2229] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:10.460615 18180 raft_consensus.cc:2258] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:10.460798 18181 tablet_replica.cc:330] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca: stopping tablet replica
I20260810 06:45:10.461117 18180 ts_tablet_manager.cc:1905] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:10.461320 18182 tablet_replica.cc:330] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7: stopping tablet replica
I20260810 06:45:10.461485 18182 raft_consensus.cc:2229] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:10.461674 18182 raft_consensus.cc:2258] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:10.461874 17895 tablet_service.cc:1467] Processing CreateTablet for tablet 2b899ea5753e49cdb04a2f73e9fa548e (DEFAULT_TABLE table=test-table [id=0f2e0172a6024afea2de4cf10570585b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:10.462020 17895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b899ea5753e49cdb04a2f73e9fa548e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:10.462347 18182 ts_tablet_manager.cc:1905] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:10.463114 18180 ts_tablet_manager.cc:1918] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:10.463230 18180 log.cc:1196] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/970d204e9a604672af08fab2803aacc6
I20260810 06:45:10.463527 18180 ts_tablet_manager.cc:1939] T 970d204e9a604672af08fab2803aacc6 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Deleting consensus metadata
I20260810 06:45:10.463825 18052 tablet_service.cc:1467] Processing CreateTablet for tablet 2b899ea5753e49cdb04a2f73e9fa548e (DEFAULT_TABLE table=test-table [id=0f2e0172a6024afea2de4cf10570585b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:10.463956 18052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b899ea5753e49cdb04a2f73e9fa548e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:10.464738 18182 ts_tablet_manager.cc:1918] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:10.464835 18182 log.cc:1196] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/wal/wals/970d204e9a604672af08fab2803aacc6
I20260810 06:45:10.465111 18182 ts_tablet_manager.cc:1939] T 970d204e9a604672af08fab2803aacc6 P c5da9a732f9b40758b9f3c9e7501f8e7: Deleting consensus metadata
I20260810 06:45:10.465480 18159 tablet_bootstrap.cc:492] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca: Bootstrap starting.
I20260810 06:45:10.465942 18159 tablet_bootstrap.cc:654] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:10.466346 18158 tablet_bootstrap.cc:492] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Bootstrap starting.
I20260810 06:45:10.466785 18158 tablet_bootstrap.cc:654] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:10.467519 17564 catalog_manager.cc:4841] TS c5da9a732f9b40758b9f3c9e7501f8e7 (127.15.232.193:37149): tablet 970d204e9a604672af08fab2803aacc6 (table test-table [id=095bf0d58159415584c95de88df238a9]) successfully deleted
I20260810 06:45:10.467592 17737 tablet_service.cc:1467] Processing CreateTablet for tablet 2b899ea5753e49cdb04a2f73e9fa548e (DEFAULT_TABLE table=test-table [id=0f2e0172a6024afea2de4cf10570585b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:10.467722 17737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b899ea5753e49cdb04a2f73e9fa548e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:10.467971 17562 catalog_manager.cc:4841] TS 2eaa13ea68cd43a2b3d27ffe950ab2f8 (127.15.232.195:43287): tablet 970d204e9a604672af08fab2803aacc6 (table test-table [id=095bf0d58159415584c95de88df238a9]) successfully deleted
I20260810 06:45:10.468175 18158 tablet_bootstrap.cc:492] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8: No bootstrap required, opened a new log
I20260810 06:45:10.468216 18158 ts_tablet_manager.cc:1397] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 06:45:10.468358 18158 raft_consensus.cc:348] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.468411 18158 raft_consensus.cc:374] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:10.468427 18158 raft_consensus.cc:729] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eaa13ea68cd43a2b3d27ffe950ab2f8, State: Initialized, Role: FOLLOWER
I20260810 06:45:10.468492 18158 consensus_queue.cc:260] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.468608 18158 ts_tablet_manager.cc:1428] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 06:45:10.468953 18159 tablet_bootstrap.cc:492] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca: No bootstrap required, opened a new log
I20260810 06:45:10.469013 18159 ts_tablet_manager.cc:1397] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 06:45:10.469111 18157 tablet_bootstrap.cc:492] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7: Bootstrap starting.
I20260810 06:45:10.469221 18159 raft_consensus.cc:348] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.469326 18159 raft_consensus.cc:374] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:10.469352 18159 raft_consensus.cc:729] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa096ac601144d6877084cb980cc4ca, State: Initialized, Role: FOLLOWER
I20260810 06:45:10.469430 18159 consensus_queue.cc:260] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.469529 18159 ts_tablet_manager.cc:1428] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:10.469555 18157 tablet_bootstrap.cc:654] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:10.470093 18181 raft_consensus.cc:2229] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:10.470450 18181 raft_consensus.cc:2258] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:10.470749 18157 tablet_bootstrap.cc:492] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7: No bootstrap required, opened a new log
I20260810 06:45:10.470799 18157 ts_tablet_manager.cc:1397] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:45:10.470971 18157 raft_consensus.cc:348] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.471014 18181 ts_tablet_manager.cc:1905] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:10.471040 18157 raft_consensus.cc:374] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:10.471062 18157 raft_consensus.cc:729] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5da9a732f9b40758b9f3c9e7501f8e7, State: Initialized, Role: FOLLOWER
I20260810 06:45:10.471114 18157 consensus_queue.cc:260] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.471220 18157 ts_tablet_manager.cc:1428] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:10.473364 18181 ts_tablet_manager.cc:1918] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:10.473433 18181 log.cc:1196] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/970d204e9a604672af08fab2803aacc6
I20260810 06:45:10.473709 18181 ts_tablet_manager.cc:1939] T 970d204e9a604672af08fab2803aacc6 P 1fa096ac601144d6877084cb980cc4ca: Deleting consensus metadata
I20260810 06:45:10.474210 17563 catalog_manager.cc:4841] TS 1fa096ac601144d6877084cb980cc4ca (127.15.232.194:33397): tablet 970d204e9a604672af08fab2803aacc6 (table test-table [id=095bf0d58159415584c95de88df238a9]) successfully deleted
I20260810 06:45:10.513820 18164 raft_consensus.cc:482] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:10.513944 18164 raft_consensus.cc:504] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.514304 18164 leader_election.cc:288] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fa096ac601144d6877084cb980cc4ca (127.15.232.194:33397), 2eaa13ea68cd43a2b3d27ffe950ab2f8 (127.15.232.195:43287)
I20260810 06:45:10.517918 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.514664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:45681 (local address 127.15.232.194:33397)
0810 06:45:10.514724 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.514727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.514875 (+ 148us) server_negotiation.cc:407] Connection header received
0810 06:45:10.514938 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.514938 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.514950 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.514987 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:10.515214 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.515979 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.517176 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.517523 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.517611 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.517622 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.517799 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.517848 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:45:10.517923 18186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.514555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33397 (local address 127.15.232.193:45681)
0810 06:45:10.514777 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.514789 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.514864 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.515075 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.515076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.515077 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:10.515132 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.515134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.516080 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.516081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.517053 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.517056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.517632 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.517633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.517727 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.517731 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.517789 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.517830 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":79,"threads_started":1}
I20260810 06:45:10.518620 17916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b899ea5753e49cdb04a2f73e9fa548e" candidate_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fa096ac601144d6877084cb980cc4ca" is_pre_election: true
I20260810 06:45:10.518802 17916 raft_consensus.cc:2454] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5da9a732f9b40758b9f3c9e7501f8e7 in term 0.
I20260810 06:45:10.518844 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.514555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:43287 (local address 127.15.232.193:57637)
0810 06:45:10.514778 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.514789 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.514868 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.515046 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.515047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.515049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:10.515132 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.515136 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.516461 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.516462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.517638 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.517642 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.518565 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.518567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.518642 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.518647 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.518708 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.518748 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":50,"thread_start_us":73,"threads_started":1}
I20260810 06:45:10.518942 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.514718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:57637 (local address 127.15.232.195:43287)
0810 06:45:10.514763 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.514765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.514907 (+ 142us) server_negotiation.cc:407] Connection header received
0810 06:45:10.514939 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.514940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.514952 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.515028 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:10.515620 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.516359 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.517733 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.518429 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.518563 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.518576 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.518726 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.518847 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:45:10.519223 18075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b899ea5753e49cdb04a2f73e9fa548e" candidate_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" is_pre_election: true
I20260810 06:45:10.519248 17681 leader_election.cc:302] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa096ac601144d6877084cb980cc4ca, c5da9a732f9b40758b9f3c9e7501f8e7; no voters:
I20260810 06:45:10.519353 18075 raft_consensus.cc:2454] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5da9a732f9b40758b9f3c9e7501f8e7 in term 0.
I20260810 06:45:10.519399 18164 raft_consensus.cc:2781] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:10.519430 18164 raft_consensus.cc:482] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:10.519449 18164 raft_consensus.cc:3037] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:10.520359 18164 raft_consensus.cc:504] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.520503 18164 leader_election.cc:288] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 1fa096ac601144d6877084cb980cc4ca (127.15.232.194:33397), 2eaa13ea68cd43a2b3d27ffe950ab2f8 (127.15.232.195:43287)
I20260810 06:45:10.520727 17916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b899ea5753e49cdb04a2f73e9fa548e" candidate_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fa096ac601144d6877084cb980cc4ca"
I20260810 06:45:10.520800 17916 raft_consensus.cc:3037] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:10.520787 18075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b899ea5753e49cdb04a2f73e9fa548e" candidate_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8"
I20260810 06:45:10.520907 18075 raft_consensus.cc:3037] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:10.521624 17916 raft_consensus.cc:2454] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5da9a732f9b40758b9f3c9e7501f8e7 in term 1.
I20260810 06:45:10.521754 18075 raft_consensus.cc:2454] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5da9a732f9b40758b9f3c9e7501f8e7 in term 1.
I20260810 06:45:10.521806 17681 leader_election.cc:302] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa096ac601144d6877084cb980cc4ca, c5da9a732f9b40758b9f3c9e7501f8e7; no voters:
I20260810 06:45:10.521881 18164 raft_consensus.cc:2781] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:10.522020 18164 raft_consensus.cc:686] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 1 LEADER]: Becoming Leader. State: Replica: c5da9a732f9b40758b9f3c9e7501f8e7, State: Running, Role: LEADER
I20260810 06:45:10.522130 18164 consensus_queue.cc:237] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.522905 17583 catalog_manager.cc:5495] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5da9a732f9b40758b9f3c9e7501f8e7 (127.15.232.193). New cstate: current_term: 1 leader_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } health_report { overall_health: UNKNOWN } } }
Aug 10 06:45:10 dist-test-slave-vxj2 krb5kdc[17529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344309, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:10.558533 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.550056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34133 (local address 127.0.0.1:51870)
0810 06:45:10.550293 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.550303 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.550365 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.550566 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.550567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.550868 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:10.550920 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.550922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.551851 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.551852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.552375 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.552376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.552842 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.552843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.552883 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.553522 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:10.553527 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:10.555281 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:10.557374 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:10.557383 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:10.557385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:10.557607 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:10.557849 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:10.557851 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:10.557851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:10.557914 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:10.558235 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:10.558242 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:10.558294 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.558354 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.558453 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":132,"threads_started":1}
I20260810 06:45:10.558678 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.550141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51870 (local address 127.15.232.254:34133)
0810 06:45:10.550225 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.550227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.550427 (+ 200us) server_negotiation.cc:407] Connection header received
0810 06:45:10.550468 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.550469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.550483 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.550554 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:10.551023 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.551758 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.552467 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.552755 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.552847 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.555384 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:10.555386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:10.555386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:10.555395 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:10.557261 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.557689 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.557690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.557690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.557768 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.557982 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.557983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.557984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.558144 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:10.558248 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.558422 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.558544 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:45:10.561004 17583 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:51870:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:45:10.561375 17583 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:10.563068 17737 tablet_service.cc:1467] Processing CreateTablet for tablet 238440866b52424982cac9100f88aef4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cb92c0de8934c15b4d6c598239e321f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:45:10.563272 17737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 238440866b52424982cac9100f88aef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:10.564347 18052 tablet_service.cc:1467] Processing CreateTablet for tablet 238440866b52424982cac9100f88aef4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cb92c0de8934c15b4d6c598239e321f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:45:10.564493 18052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 238440866b52424982cac9100f88aef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:10.565095 18157 tablet_bootstrap.cc:492] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7: Bootstrap starting.
I20260810 06:45:10.565246 17895 tablet_service.cc:1467] Processing CreateTablet for tablet 238440866b52424982cac9100f88aef4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cb92c0de8934c15b4d6c598239e321f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:45:10.565407 17895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 238440866b52424982cac9100f88aef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:10.565619 18157 tablet_bootstrap.cc:654] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:10.567006 18159 tablet_bootstrap.cc:492] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: Bootstrap starting.
I20260810 06:45:10.567682 18158 tablet_bootstrap.cc:492] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Bootstrap starting.
I20260810 06:45:10.567770 18159 tablet_bootstrap.cc:654] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:10.568040 18158 tablet_bootstrap.cc:654] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:10.568718 18158 tablet_bootstrap.cc:492] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: No bootstrap required, opened a new log
I20260810 06:45:10.568794 18158 ts_tablet_manager.cc:1397] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:10.568841 18157 tablet_bootstrap.cc:492] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7: No bootstrap required, opened a new log
I20260810 06:45:10.568912 18157 ts_tablet_manager.cc:1397] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260810 06:45:10.568979 18158 raft_consensus.cc:348] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.569082 18158 raft_consensus.cc:374] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:10.569110 18158 raft_consensus.cc:729] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eaa13ea68cd43a2b3d27ffe950ab2f8, State: Initialized, Role: FOLLOWER
I20260810 06:45:10.569157 18158 consensus_queue.cc:260] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.569289 18158 ts_tablet_manager.cc:1428] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 06:45:10.569418 18159 tablet_bootstrap.cc:492] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: No bootstrap required, opened a new log
I20260810 06:45:10.569484 18159 ts_tablet_manager.cc:1397] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:45:10.569535 18163 tablet_replica.cc:439] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } }
I20260810 06:45:10.569648 18163 tablet_replica.cc:442] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:10.569660 18159 raft_consensus.cc:348] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.569757 18159 raft_consensus.cc:374] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:10.569790 18159 raft_consensus.cc:729] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa096ac601144d6877084cb980cc4ca, State: Initialized, Role: FOLLOWER
I20260810 06:45:10.569854 18159 consensus_queue.cc:260] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.569975 18159 ts_tablet_manager.cc:1428] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:10.570011 18167 tablet_replica.cc:439] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } }
I20260810 06:45:10.570111 18167 tablet_replica.cc:442] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:10.569095 18157 raft_consensus.cc:348] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.570571 18157 raft_consensus.cc:374] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:10.570624 18157 raft_consensus.cc:729] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5da9a732f9b40758b9f3c9e7501f8e7, State: Initialized, Role: FOLLOWER
I20260810 06:45:10.570688 18157 consensus_queue.cc:260] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.570802 18157 ts_tablet_manager.cc:1428] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260810 06:45:10.570932 18164 tablet_replica.cc:439] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } }
I20260810 06:45:10.571010 18164 tablet_replica.cc:442] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:10.654575 18167 raft_consensus.cc:482] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:10.654707 18167 raft_consensus.cc:504] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.654933 18167 leader_election.cc:288] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5da9a732f9b40758b9f3c9e7501f8e7 (127.15.232.193:37149), 2eaa13ea68cd43a2b3d27ffe950ab2f8 (127.15.232.195:43287)
I20260810 06:45:10.655282 18075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "238440866b52424982cac9100f88aef4" candidate_uuid: "1fa096ac601144d6877084cb980cc4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" is_pre_election: true
I20260810 06:45:10.655285 17759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "238440866b52424982cac9100f88aef4" candidate_uuid: "1fa096ac601144d6877084cb980cc4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" is_pre_election: true
I20260810 06:45:10.655386 18075 raft_consensus.cc:2454] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fa096ac601144d6877084cb980cc4ca in term 0.
I20260810 06:45:10.655411 17759 raft_consensus.cc:2454] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fa096ac601144d6877084cb980cc4ca in term 0.
I20260810 06:45:10.655597 17843 leader_election.cc:302] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa096ac601144d6877084cb980cc4ca, 2eaa13ea68cd43a2b3d27ffe950ab2f8; no voters:
I20260810 06:45:10.655738 18167 raft_consensus.cc:2781] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:10.655833 18167 raft_consensus.cc:482] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:10.655850 18167 raft_consensus.cc:3037] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:10.656579 18167 raft_consensus.cc:504] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.656730 18167 leader_election.cc:288] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [CANDIDATE]: Term 1 election: Requested vote from peers c5da9a732f9b40758b9f3c9e7501f8e7 (127.15.232.193:37149), 2eaa13ea68cd43a2b3d27ffe950ab2f8 (127.15.232.195:43287)
I20260810 06:45:10.656883 17759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "238440866b52424982cac9100f88aef4" candidate_uuid: "1fa096ac601144d6877084cb980cc4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7"
I20260810 06:45:10.656991 17759 raft_consensus.cc:3037] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:10.656998 18075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "238440866b52424982cac9100f88aef4" candidate_uuid: "1fa096ac601144d6877084cb980cc4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8"
I20260810 06:45:10.657097 18075 raft_consensus.cc:3037] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:10.657684 17759 raft_consensus.cc:2454] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fa096ac601144d6877084cb980cc4ca in term 1.
I20260810 06:45:10.657775 18075 raft_consensus.cc:2454] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fa096ac601144d6877084cb980cc4ca in term 1.
I20260810 06:45:10.657876 17845 leader_election.cc:302] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa096ac601144d6877084cb980cc4ca, c5da9a732f9b40758b9f3c9e7501f8e7; no voters:
I20260810 06:45:10.658016 18167 raft_consensus.cc:2781] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:10.658087 18167 raft_consensus.cc:686] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [term 1 LEADER]: Becoming Leader. State: Replica: 1fa096ac601144d6877084cb980cc4ca, State: Running, Role: LEADER
I20260810 06:45:10.658152 18167 consensus_queue.cc:237] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } }
I20260810 06:45:10.658463 18165 tablet_replica.cc:439] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: TxnStatusTablet state changed. Reason: New leader 1fa096ac601144d6877084cb980cc4ca. Latest consensus state: current_term: 1 leader_uuid: "1fa096ac601144d6877084cb980cc4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } }
I20260810 06:45:10.658560 18165 tablet_replica.cc:442] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:45:10.658739 18202 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:45:10.658901 17583 catalog_manager.cc:5495] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fa096ac601144d6877084cb980cc4ca (127.15.232.194). New cstate: current_term: 1 leader_uuid: "1fa096ac601144d6877084cb980cc4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } health_report { overall_health: UNKNOWN } } }
I20260810 06:45:10.660156 18202 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:45:10.668177 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.665834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.15.232.194:33397)
0810 06:45:10.665883 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.665885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.665895 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:10.665983 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.665983 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.665994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.666043 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:10.666158 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.666651 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.667454 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.667668 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.667754 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.667945 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:10.667985 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:10.668034 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.668090 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.668110 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:45:10.668219 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.665772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33397 (local address 127.0.0.1:60136)
0810 06:45:10.665847 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.665860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.665915 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.666047 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.666048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.666049 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:10.666102 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.666104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.666761 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.666762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.667362 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.667364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.667754 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.667755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.667830 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.667846 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:10.668037 (+ 191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:10.668043 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.668089 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.668139 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:45:10.669536 18075 raft_consensus.cc:1261] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 1 FOLLOWER]: Refusing update from remote peer 1fa096ac601144d6877084cb980cc4ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:10.669643 17759 raft_consensus.cc:1261] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7 [term 1 FOLLOWER]: Refusing update from remote peer 1fa096ac601144d6877084cb980cc4ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:10.669852 18165 consensus_queue.cc:1035] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:10.669971 18167 consensus_queue.cc:1035] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:10.670501 18164 tablet_replica.cc:439] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7: TxnStatusTablet state changed. Reason: New leader 1fa096ac601144d6877084cb980cc4ca. Latest consensus state: current_term: 1 leader_uuid: "1fa096ac601144d6877084cb980cc4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } }
I20260810 06:45:10.670496 18163 tablet_replica.cc:439] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: TxnStatusTablet state changed. Reason: New leader 1fa096ac601144d6877084cb980cc4ca. Latest consensus state: current_term: 1 leader_uuid: "1fa096ac601144d6877084cb980cc4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } }
I20260810 06:45:10.670591 18163 tablet_replica.cc:442] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:10.670591 18164 tablet_replica.cc:442] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:10.671486 18165 tablet_replica.cc:439] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fa096ac601144d6877084cb980cc4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } }
I20260810 06:45:10.671588 18165 tablet_replica.cc:442] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:45:10.671494 18206 tablet_replica.cc:439] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fa096ac601144d6877084cb980cc4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } }
I20260810 06:45:10.671672 18206 tablet_replica.cc:442] T 238440866b52424982cac9100f88aef4 P 1fa096ac601144d6877084cb980cc4ca: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:45:10.672128 18163 tablet_replica.cc:439] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fa096ac601144d6877084cb980cc4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } }
I20260810 06:45:10.672217 18163 tablet_replica.cc:442] T 238440866b52424982cac9100f88aef4 P 2eaa13ea68cd43a2b3d27ffe950ab2f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:10.672327 18173 mvcc.cc:205] Tried to move back new op lower bound from 7316866296500592640 to 7316866296456232960. Current Snapshot: MvccSnapshot[applied={T|T < 7316866296500592640}]
I20260810 06:45:10.672410 18164 tablet_replica.cc:439] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fa096ac601144d6877084cb980cc4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5da9a732f9b40758b9f3c9e7501f8e7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37149 } } peers { permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 } } peers { permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 } } }
I20260810 06:45:10.672497 18164 tablet_replica.cc:442] T 238440866b52424982cac9100f88aef4 P c5da9a732f9b40758b9f3c9e7501f8e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:10.677125 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.674686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56410 (local address 127.15.232.193:37149)
0810 06:45:10.674734 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.674736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.674746 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:10.674841 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.674842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.674862 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.674946 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:10.675076 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.675569 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.676328 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.676541 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.676621 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.676765 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:10.676882 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:10.676971 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.677024 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.677044 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:45:10.683003 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.680170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:33367 (local address 127.15.232.194:33397)
0810 06:45:10.680248 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.680250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.680279 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:45:10.680513 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.680514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.680527 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.680581 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:10.680828 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.681368 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.682261 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.682484 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.682564 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.682751 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:10.682788 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:10.682849 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.682912 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.682930 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:45:10.683037 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.680009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33397 (local address 127.15.232.193:33367)
0810 06:45:10.680214 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.680227 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.680297 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.680592 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.680593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.680595 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:10.680672 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.680673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.681471 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.681472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.682156 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.682158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.682565 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.682566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.682628 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.682651 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:10.682852 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:10.682855 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.682907 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.682953 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
I20260810 06:45:10.685350 18075 raft_consensus.cc:1261] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 1 FOLLOWER]: Refusing update from remote peer c5da9a732f9b40758b9f3c9e7501f8e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:10.685367 17916 raft_consensus.cc:1261] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca [term 1 FOLLOWER]: Refusing update from remote peer c5da9a732f9b40758b9f3c9e7501f8e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:10.685606 18188 consensus_queue.cc:1035] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2eaa13ea68cd43a2b3d27ffe950ab2f8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 43287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:10.685716 18164 consensus_queue.cc:1035] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fa096ac601144d6877084cb980cc4ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:10.693876 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.691579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:47655 (local address 127.15.232.193:37149)
0810 06:45:10.691614 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.691617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.691625 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:10.691671 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.691672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.691682 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.691742 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:10.691851 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.692338 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.693103 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.693367 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.693442 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.693643 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:10.693677 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:10.693723 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.693790 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.693809 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 06:45:10.693931 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.691362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:37149 (local address 127.15.232.194:47655)
0810 06:45:10.691548 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:10.691560 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:10.691615 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:10.691745 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:10.691746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:10.691747 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:10.691794 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.691796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.692422 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.692423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.693007 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:10.693009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.693441 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.693442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:10.693500 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.693517 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:10.693732 (+ 215us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:10.693734 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:45:10.693785 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 06:45:10.693829 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":112,"threads_started":1}
I20260810 06:45:10.704005 17582 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:10.704133 17582 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:10.704747 17582 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36fd495f17b240aeb3d629b908dc6a54: Sending DeleteTablet for 3 replicas of tablet 2b899ea5753e49cdb04a2f73e9fa548e
I20260810 06:45:10.705063 17737 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b899ea5753e49cdb04a2f73e9fa548e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:10 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:56396
I20260810 06:45:10.705118 17895 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b899ea5753e49cdb04a2f73e9fa548e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:10 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:60130
I20260810 06:45:10.705210 18052 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b899ea5753e49cdb04a2f73e9fa548e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:10 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:33150
I20260810 06:45:10.705246 18182 tablet_replica.cc:330] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7: stopping tablet replica
I20260810 06:45:10.705299 18182 raft_consensus.cc:2229] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:10.705304 18181 tablet_replica.cc:330] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca: stopping tablet replica
I20260810 06:45:10.705320 18180 tablet_replica.cc:330] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8: stopping tablet replica
I20260810 06:45:10.705368 18181 raft_consensus.cc:2229] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:10.705389 18180 raft_consensus.cc:2229] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:10.705389 18182 raft_consensus.cc:2258] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:10.705401 18181 raft_consensus.cc:2258] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:10.705430 18180 raft_consensus.cc:2258] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:10.705798 18180 ts_tablet_manager.cc:1905] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:10.705919 18182 ts_tablet_manager.cc:1905] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:10.706579 18181 ts_tablet_manager.cc:1905] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:10.707911 18182 ts_tablet_manager.cc:1918] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:10.707965 18182 log.cc:1196] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-0/wal/wals/2b899ea5753e49cdb04a2f73e9fa548e
I20260810 06:45:10.708189 18182 ts_tablet_manager.cc:1939] T 2b899ea5753e49cdb04a2f73e9fa548e P c5da9a732f9b40758b9f3c9e7501f8e7: Deleting consensus metadata
I20260810 06:45:10.708354 18181 ts_tablet_manager.cc:1918] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:10.708407 18181 log.cc:1196] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/2b899ea5753e49cdb04a2f73e9fa548e
I20260810 06:45:10.708591 18181 ts_tablet_manager.cc:1939] T 2b899ea5753e49cdb04a2f73e9fa548e P 1fa096ac601144d6877084cb980cc4ca: Deleting consensus metadata
I20260810 06:45:10.708884 17563 catalog_manager.cc:4841] TS 1fa096ac601144d6877084cb980cc4ca (127.15.232.194:33397): tablet 2b899ea5753e49cdb04a2f73e9fa548e (table test-table [id=0f2e0172a6024afea2de4cf10570585b]) successfully deleted
I20260810 06:45:10.708948 17564 catalog_manager.cc:4841] TS c5da9a732f9b40758b9f3c9e7501f8e7 (127.15.232.193:37149): tablet 2b899ea5753e49cdb04a2f73e9fa548e (table test-table [id=0f2e0172a6024afea2de4cf10570585b]) successfully deleted
I20260810 06:45:10.709106 18180 ts_tablet_manager.cc:1918] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:10.709199 18180 log.cc:1196] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/2b899ea5753e49cdb04a2f73e9fa548e
I20260810 06:45:10.709410 18180 ts_tablet_manager.cc:1939] T 2b899ea5753e49cdb04a2f73e9fa548e P 2eaa13ea68cd43a2b3d27ffe950ab2f8: Deleting consensus metadata
I20260810 06:45:10.709733 17562 catalog_manager.cc:4841] TS 2eaa13ea68cd43a2b3d27ffe950ab2f8 (127.15.232.195:43287): tablet 2b899ea5753e49cdb04a2f73e9fa548e (table test-table [id=0f2e0172a6024afea2de4cf10570585b]) successfully deleted
Aug 10 06:45:10 dist-test-slave-vxj2 krb5kdc[17529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344310, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.232.193@KRBTEST.COM
I20260810 06:45:10.722906 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.713495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56422 (local address 127.15.232.193:37149)
0810 06:45:10.713554 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.713556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.713661 (+ 105us) server_negotiation.cc:407] Connection header received
0810 06:45:10.713733 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.713734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.713747 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.713812 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:10.714450 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.714985 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.715783 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.715997 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.716089 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.719449 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:10.719459 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:10.719459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:10.719469 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:10.721392 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.721810 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.721820 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.721821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.721883 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.722138 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.722139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.722140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.722348 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:10.722592 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.722637 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.722740 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260810 06:45:10.723291 17791 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56422
I20260810 06:45:10.732458 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:10.724482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51884 (local address 127.15.232.254:34133)
0810 06:45:10.724537 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:10.724539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:10.725326 (+ 787us) server_negotiation.cc:407] Connection header received
0810 06:45:10.725391 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:10.725393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:10.725406 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:10.725469 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:10.726121 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.726653 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.727399 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:10.727614 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:10.727707 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:10.729008 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:10.729010 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:10.729010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:10.729019 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:10.731091 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.731558 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.731559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.731559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.731630 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:10.731868 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:10.731869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:10.731870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:10.732001 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:10.732119 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:10.732242 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 06:45:10.732332 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260810 06:45:10.732721 17582 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51884
I20260810 06:45:10.733732 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 17658
I20260810 06:45:10.741874 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 17824
I20260810 06:45:10.751114 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 17978
I20260810 06:45:10.758678 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 17545
2026-08-10T06:45:10Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2535 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:10 dist-test-slave-vxj2 krb5kdc[18224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:10 dist-test-slave-vxj2 krb5kdc[18224](info): set up 2 sockets
Aug 10 06:45:10 dist-test-slave-vxj2 krb5kdc[18224](info): commencing operation
krb5kdc: starting...
W20260810 06:45:11.907223 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:11 dist-test-slave-vxj2 krb5kdc[18224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:45:12.037050 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:42065
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:42065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:12.122388 18238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:12.122663 18238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:12.122714 18238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:12.123982 18238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:12.124073 18238 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:45:12.124099 18238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:12.124123 18238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:12.124159 18238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:12.124185 18238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:12.125818 18238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:42065
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:42065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:12.126142 18238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:12.126407 18238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:12.131637 18243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:12.131655 18246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:12.131644 18244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:12.132107 18238 server_base.cc:1034] running on GCE node
I20260810 06:45:12.132409 18238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:12.132791 18238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:12.134008 18238 hybrid_clock.cc:648] HybridClock initialized: now 1786344312134006 us; error 63 us; skew 500 ppm
Aug 10 06:45:12 dist-test-slave-vxj2 krb5kdc[18224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344312, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:12.136863 18238 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:45:12.137436 18238 webserver.cc:460] Webserver started at http://127.15.232.254:44279/ using document root <none> and password file <none>
I20260810 06:45:12.137718 18238 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:12.137792 18238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:12.137964 18238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:12.139266 18238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "c0f6671e9af244c8adbc4d96c888c256"
format_stamp: "Formatted at 2026-08-10 06:45:12 on dist-test-slave-vxj2"
server_key: "beaae8bb5fa2e294aa4554208df6715f"
server_key_iv: "69e243441b576f576c8d35e8ea4e4e50"
server_key_version: "encryptionkey@0"
I20260810 06:45:12.139667 18238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "c0f6671e9af244c8adbc4d96c888c256"
format_stamp: "Formatted at 2026-08-10 06:45:12 on dist-test-slave-vxj2"
server_key: "beaae8bb5fa2e294aa4554208df6715f"
server_key_iv: "69e243441b576f576c8d35e8ea4e4e50"
server_key_version: "encryptionkey@0"
I20260810 06:45:12.141136 18238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:12.142185 18253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:12.142436 18238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:12.142570 18238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "c0f6671e9af244c8adbc4d96c888c256"
format_stamp: "Formatted at 2026-08-10 06:45:12 on dist-test-slave-vxj2"
server_key: "beaae8bb5fa2e294aa4554208df6715f"
server_key_iv: "69e243441b576f576c8d35e8ea4e4e50"
server_key_version: "encryptionkey@0"
I20260810 06:45:12.142660 18238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:12.161829 18238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:12.164366 18238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:12.164551 18238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:12.169644 18238 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:42065
I20260810 06:45:12.169713 18325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:42065 every 8 connection(s)
I20260810 06:45:12.170112 18238 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:12.170826 18326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:12.172839 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 18238
I20260810 06:45:12.172968 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:12.173195 16291 external_mini_cluster.cc:1428] Setting key 9480c2917588c8be806f7e0aa7dc5b75
I20260810 06:45:12.173445 18326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256: Bootstrap starting.
I20260810 06:45:12.174163 18326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:12.174477 18326 log.cc:824] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:12.175338 18326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256: No bootstrap required, opened a new log
I20260810 06:45:12.177623 18326 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f6671e9af244c8adbc4d96c888c256" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42065 } }
I20260810 06:45:12.177829 18326 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:12.177862 18326 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f6671e9af244c8adbc4d96c888c256, State: Initialized, Role: FOLLOWER
I20260810 06:45:12.178038 18326 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f6671e9af244c8adbc4d96c888c256" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42065 } }
I20260810 06:45:12.178103 18326 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:12.178145 18326 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:12.178194 18326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:12.178910 18326 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f6671e9af244c8adbc4d96c888c256" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42065 } }
I20260810 06:45:12.179006 18326 leader_election.cc:302] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0f6671e9af244c8adbc4d96c888c256; no voters:
I20260810 06:45:12.179241 18326 leader_election.cc:288] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:12.179316 18332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:12.179549 18332 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [term 1 LEADER]: Becoming Leader. State: Replica: c0f6671e9af244c8adbc4d96c888c256, State: Running, Role: LEADER
I20260810 06:45:12.179584 18326 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:12.179659 18332 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f6671e9af244c8adbc4d96c888c256" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42065 } }
I20260810 06:45:12.179973 18332 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0f6671e9af244c8adbc4d96c888c256. Latest consensus state: current_term: 1 leader_uuid: "c0f6671e9af244c8adbc4d96c888c256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f6671e9af244c8adbc4d96c888c256" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42065 } } }
I20260810 06:45:12.180069 18332 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:12.179986 18334 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0f6671e9af244c8adbc4d96c888c256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f6671e9af244c8adbc4d96c888c256" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42065 } } }
I20260810 06:45:12.180228 18334 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:12.180351 18338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:12.180927 18338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:45:12 dist-test-slave-vxj2 krb5kdc[18224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:12.182794 18338 catalog_manager.cc:1347] Generated new cluster ID: c7a018eaede64c2992d2f1a2d4b766d7
I20260810 06:45:12.182889 18338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:12.186299 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.173870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49932 (local address 127.15.232.254:42065)
0810 06:45:12.174067 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:12.174077 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:12.174116 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:45:12.174663 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:12.174689 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:12.174764 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:12.174925 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:12.175806 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.176671 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.177864 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.178473 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.178563 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.181835 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:12.181839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:12.181839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:12.181853 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:12.184020 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.184473 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.184476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.184476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.184585 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.184805 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.184806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.184806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.185176 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:12.185345 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:12.185922 (+ 577us) server_negotiation.cc:299] Negotiation successful
0810 06:45:12.186074 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260810 06:45:12.198416 18338 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:12.199160 18338 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:12.211028 18338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256: Generated new TSK 0
I20260810 06:45:12.211298 18338 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:45:12.240628 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:42065
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:12.327217 18355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:12.327450 18355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:12.327476 18355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:12.328706 18355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:12.328754 18355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:12.328778 18355 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:45:12.328788 18355 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:45:12.328809 18355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:12.330418 18355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.254:42065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:12.330699 18355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:12.330951 18355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:12.336477 18363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:12.336477 18366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:12.336520 18355 server_base.cc:1034] running on GCE node
W20260810 06:45:12.336517 18362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:12.337054 18355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:12.337363 18355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:12.338546 18355 hybrid_clock.cc:648] HybridClock initialized: now 1786344312338529 us; error 45 us; skew 500 ppm
Aug 10 06:45:12 dist-test-slave-vxj2 krb5kdc[18224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344312, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:12.341446 18355 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:45:12.341872 18355 webserver.cc:460] Webserver started at http://127.15.232.193:37147/ using document root <none> and password file <none>
I20260810 06:45:12.342079 18355 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:12.342120 18355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:12.342212 18355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:12.343245 18355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "f598e218e9f3407983f409a6d100404b"
format_stamp: "Formatted at 2026-08-10 06:45:12 on dist-test-slave-vxj2"
server_key: "8643fc5832b6457bd5e503ab13e266be"
server_key_iv: "dc30726209dd572d322b2af82e6cc608"
server_key_version: "encryptionkey@0"
I20260810 06:45:12.343561 18355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "f598e218e9f3407983f409a6d100404b"
format_stamp: "Formatted at 2026-08-10 06:45:12 on dist-test-slave-vxj2"
server_key: "8643fc5832b6457bd5e503ab13e266be"
server_key_iv: "dc30726209dd572d322b2af82e6cc608"
server_key_version: "encryptionkey@0"
I20260810 06:45:12.344861 18355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:12.345810 18374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:12.346006 18355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:45:12.346107 18355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "f598e218e9f3407983f409a6d100404b"
format_stamp: "Formatted at 2026-08-10 06:45:12 on dist-test-slave-vxj2"
server_key: "8643fc5832b6457bd5e503ab13e266be"
server_key_iv: "dc30726209dd572d322b2af82e6cc608"
server_key_version: "encryptionkey@0"
I20260810 06:45:12.346166 18355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:12.381008 18355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:12.383801 18355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:12.383940 18355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:12.384491 18355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:12.384532 18355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:12.384557 18355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:12.384624 18355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:45:12 dist-test-slave-vxj2 krb5kdc[18224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344312, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:12.392681 18355 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:42985
I20260810 06:45:12.392931 18504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:42985 every 8 connection(s)
I20260810 06:45:12.393209 18355 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:12.396250 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.385174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:59229 (local address 127.15.232.254:42065)
0810 06:45:12.385245 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:12.385248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:12.385544 (+ 296us) server_negotiation.cc:407] Connection header received
0810 06:45:12.386232 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:12.386234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:12.386250 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:12.386334 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:12.387448 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.388430 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.389783 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.390149 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.390229 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.393269 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:12.393270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:12.393271 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:12.393278 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:12.394811 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.395205 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.395206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.395206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.395288 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.395572 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.395573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.395574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.395706 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:12.395822 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:12.396076 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 06:45:12.396162 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:45:12.396260 18383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.385131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42065 (local address 127.15.232.193:59229)
0810 06:45:12.385355 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:12.385422 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:12.386085 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:12.386502 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:12.386506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:12.387003 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:12.387319 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.387325 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.388553 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.388554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.389682 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.389686 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.390259 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.390261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.390452 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.391089 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:12.391102 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:12.393163 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:12.394890 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:12.394897 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:12.394898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:12.395119 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:12.395345 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:12.395345 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:12.395346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:12.395450 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:12.395770 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:12.395774 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:12.395893 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 06:45:12.395971 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:45:12.396097 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":109,"threads_started":1}
I20260810 06:45:12.396921 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 18355
I20260810 06:45:12.396944 18506 heartbeater.cc:347] Connected to a master server at 127.15.232.254:42065
I20260810 06:45:12.397030 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:12.397033 18506 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:12.397259 18506 heartbeater.cc:511] Master 127.15.232.254:42065 requested a full tablet report, sending...
I20260810 06:45:12.397282 16291 external_mini_cluster.cc:1428] Setting key ac69d672189c6f51ffcf298139c84c94
I20260810 06:45:12.397855 18271 ts_manager.cc:194] Registered new tserver with Master: f598e218e9f3407983f409a6d100404b (127.15.232.193:42985)
I20260810 06:45:12.398653 18271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:59229
I20260810 06:45:12.405591 18512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.398101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42065 (local address 127.15.232.193:56959)
0810 06:45:12.398238 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:12.398251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:12.398305 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:12.399005 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:12.399006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:12.399240 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:12.399318 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.399320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.400054 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.400055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.400896 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.400900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.401458 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.401459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.401521 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.402000 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:12.402006 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:12.402401 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:12.404419 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:12.404420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:12.404421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:12.404591 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:12.404819 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:12.404819 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:12.404820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:12.404889 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:12.405316 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:12.405317 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:12.405363 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 06:45:12.405426 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 06:45:12.405515 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260810 06:45:12.405697 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.398135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:56959 (local address 127.15.232.254:42065)
0810 06:45:12.398860 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:12.398867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:12.398876 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:12.398909 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:12.398909 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:12.398919 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:12.398983 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:12.399415 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.399947 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.401021 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.401375 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.401460 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.402511 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:12.402513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:12.402513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:12.402522 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:12.404328 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.404669 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.404671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.404672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.404741 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.404956 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.404957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.404958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.405102 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:12.405219 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:12.405487 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 06:45:12.405580 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706}
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:45:12.427582 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:42065
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:12.510582 18516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:12.510842 18516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:12.510887 18516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:12.512104 18516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:12.512177 18516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:12.512217 18516 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:45:12.512264 18516 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:45:12.512305 18516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:12.513870 18516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.254:42065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:12.514158 18516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:12.514397 18516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:12.519671 18522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:12.519692 18525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:12.519699 18521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:12.520215 18516 server_base.cc:1034] running on GCE node
I20260810 06:45:12.520428 18516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:12.520678 18516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:12.521966 18516 hybrid_clock.cc:648] HybridClock initialized: now 1786344312521949 us; error 24 us; skew 500 ppm
Aug 10 06:45:12 dist-test-slave-vxj2 krb5kdc[18224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344312, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:12.524564 18516 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:45:12.525045 18516 webserver.cc:460] Webserver started at http://127.15.232.194:39249/ using document root <none> and password file <none>
I20260810 06:45:12.525305 18516 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:12.525353 18516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:12.525451 18516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:12.526505 18516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "8d8e1a8ac5aa48aba9a19782fe026d95"
format_stamp: "Formatted at 2026-08-10 06:45:12 on dist-test-slave-vxj2"
server_key: "6aef9e2891c5b1775f8c02d06af2fb42"
server_key_iv: "31849789f8d20b4fe1ac3471c2fba2da"
server_key_version: "encryptionkey@0"
I20260810 06:45:12.526841 18516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "8d8e1a8ac5aa48aba9a19782fe026d95"
format_stamp: "Formatted at 2026-08-10 06:45:12 on dist-test-slave-vxj2"
server_key: "6aef9e2891c5b1775f8c02d06af2fb42"
server_key_iv: "31849789f8d20b4fe1ac3471c2fba2da"
server_key_version: "encryptionkey@0"
I20260810 06:45:12.528225 18516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:12.528968 18535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:12.529206 18516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:12.529323 18516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "8d8e1a8ac5aa48aba9a19782fe026d95"
format_stamp: "Formatted at 2026-08-10 06:45:12 on dist-test-slave-vxj2"
server_key: "6aef9e2891c5b1775f8c02d06af2fb42"
server_key_iv: "31849789f8d20b4fe1ac3471c2fba2da"
server_key_version: "encryptionkey@0"
I20260810 06:45:12.529455 18516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:12.570062 18516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:12.572969 18516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:12.573136 18516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:12.573740 18516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:12.573812 18516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:12.573859 18516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:12.573879 18516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:12.580703 18516 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:45517
I20260810 06:45:12.580816 18667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:45517 every 8 connection(s)
I20260810 06:45:12.581106 18516 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:45:12 dist-test-slave-vxj2 krb5kdc[18224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344312, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:12.583724 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 18516
I20260810 06:45:12.583868 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:12.584089 16291 external_mini_cluster.cc:1428] Setting key 40c5b402bbef9b5d75a628fa40d8d168
I20260810 06:45:12.585764 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.574747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:48533 (local address 127.15.232.254:42065)
0810 06:45:12.574804 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:12.574807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:12.575099 (+ 292us) server_negotiation.cc:407] Connection header received
0810 06:45:12.575887 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:12.575889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:12.575903 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:12.575957 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:12.576930 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.577633 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.579025 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.579351 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.579448 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.582257 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:12.582259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:12.582259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:12.582266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:12.584055 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.584612 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.584613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.584614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.584669 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.584981 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.584981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.584982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.585093 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:12.585222 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:12.585614 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 06:45:12.585692 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 06:45:12.586097 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.574699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42065 (local address 127.15.232.194:48533)
0810 06:45:12.574983 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:12.575013 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:12.575759 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:12.576098 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:12.576102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:12.576502 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:12.576835 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.576842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.577737 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.577738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.578930 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.578933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.579457 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.579459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.579664 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.580313 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:12.580325 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:12.582166 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:12.584158 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:12.584167 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:12.584170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:12.584523 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:12.584746 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:12.584747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:12.584747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:12.584890 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:12.585231 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:12.585239 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:12.585434 (+ 195us) client_negotiation.cc:769] Sending connection context
0810 06:45:12.585574 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 06:45:12.585794 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":106,"threads_started":1}
I20260810 06:45:12.586721 18668 heartbeater.cc:347] Connected to a master server at 127.15.232.254:42065
I20260810 06:45:12.586848 18668 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:12.587046 18668 heartbeater.cc:511] Master 127.15.232.254:42065 requested a full tablet report, sending...
I20260810 06:45:12.587425 18271 ts_manager.cc:194] Registered new tserver with Master: 8d8e1a8ac5aa48aba9a19782fe026d95 (127.15.232.194:45517)
I20260810 06:45:12.587918 18271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:48533
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
I20260810 06:45:12.594959 18676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.588178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42065 (local address 127.15.232.194:59017)
0810 06:45:12.588344 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:12.588356 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:12.588421 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:12.588562 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:12.588563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:12.588772 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:12.588840 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.588842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.589864 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.589865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.590389 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.590391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.590962 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.590963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.591028 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.591564 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:12.591568 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:12.591864 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:12.593812 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:12.593812 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:12.593813 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:12.593989 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:12.594227 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:12.594228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:12.594228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:12.594283 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:12.594652 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:12.594653 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:12.594692 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 06:45:12.594738 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 06:45:12.594822 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":78,"threads_started":1}
I20260810 06:45:12.594990 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.588225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:59017 (local address 127.15.232.254:42065)
0810 06:45:12.588268 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:12.588270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:12.588434 (+ 164us) server_negotiation.cc:407] Connection header received
0810 06:45:12.588468 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:12.588469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:12.588482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:12.588558 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:12.588928 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.589761 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.590483 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.590850 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.590942 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.591963 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:12.591965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:12.591965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:12.591974 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:12.593724 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.594072 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.594073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.594074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.594144 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.594350 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.594351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.594351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.594573 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:12.594651 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:12.594772 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 06:45:12.594864 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:45:12.616631 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:42065
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:12.701635 18678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:12.701864 18678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:12.701891 18678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:12.703042 18678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:12.703087 18678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:12.703107 18678 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:45:12.703119 18678 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:45:12.703137 18678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:12.704602 18678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.254:42065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:12.704907 18678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:12.705111 18678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:12.710539 18683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:12.710704 18678 server_base.cc:1034] running on GCE node
W20260810 06:45:12.710747 18686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:12.710548 18684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:12.711094 18678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:12.711326 18678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:12.712474 18678 hybrid_clock.cc:648] HybridClock initialized: now 1786344312712455 us; error 40 us; skew 500 ppm
Aug 10 06:45:12 dist-test-slave-vxj2 krb5kdc[18224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344312, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:12.715210 18678 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:45:12.715638 18678 webserver.cc:460] Webserver started at http://127.15.232.195:34283/ using document root <none> and password file <none>
I20260810 06:45:12.715831 18678 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:12.715924 18678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:12.716071 18678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:12.717223 18678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "36667d5b23cc42d2b23692f7553dcee3"
format_stamp: "Formatted at 2026-08-10 06:45:12 on dist-test-slave-vxj2"
server_key: "91931343ce1ad69dd8180537fa021d51"
server_key_iv: "5d703c6fafcae1c956a1cfdcc415cb01"
server_key_version: "encryptionkey@0"
I20260810 06:45:12.717583 18678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "36667d5b23cc42d2b23692f7553dcee3"
format_stamp: "Formatted at 2026-08-10 06:45:12 on dist-test-slave-vxj2"
server_key: "91931343ce1ad69dd8180537fa021d51"
server_key_iv: "5d703c6fafcae1c956a1cfdcc415cb01"
server_key_version: "encryptionkey@0"
I20260810 06:45:12.718988 18678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:12.719842 18694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:12.720103 18678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:12.720225 18678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "36667d5b23cc42d2b23692f7553dcee3"
format_stamp: "Formatted at 2026-08-10 06:45:12 on dist-test-slave-vxj2"
server_key: "91931343ce1ad69dd8180537fa021d51"
server_key_iv: "5d703c6fafcae1c956a1cfdcc415cb01"
server_key_version: "encryptionkey@0"
I20260810 06:45:12.720307 18678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:12.734359 18678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:12.736474 18678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:12.736606 18678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:12.737304 18678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:12.737357 18678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:12.737382 18678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:12.737396 18678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:45:12 dist-test-slave-vxj2 krb5kdc[18224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344312, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:12.745347 18678 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:33757
I20260810 06:45:12.745441 18821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:33757 every 8 connection(s)
I20260810 06:45:12.745867 18678 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:12.748334 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.737785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:57069 (local address 127.15.232.254:42065)
0810 06:45:12.737863 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:12.737866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:12.738114 (+ 248us) server_negotiation.cc:407] Connection header received
0810 06:45:12.738679 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:12.738681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:12.738694 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:12.738746 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:12.739470 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.740270 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.741969 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.742372 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.742488 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.745289 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:12.745291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:12.745291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:12.745301 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:12.747090 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.747517 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.747518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.747518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.747573 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.747786 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.747787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.747787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.747899 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:12.747954 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:12.748172 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 06:45:12.748265 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:45:12.748485 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.737781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42065 (local address 127.15.232.195:57069)
0810 06:45:12.737971 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:12.738019 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:12.738553 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:12.738820 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:12.738825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:12.739100 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:12.739368 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.739374 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.740380 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.740381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.741841 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.741844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.742502 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.742504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.742658 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.743374 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:12.743391 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:12.745144 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:12.747192 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:12.747199 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:12.747201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:12.747443 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:12.747635 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:12.747636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:12.747636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:12.747730 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:12.747943 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:12.747947 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:12.748061 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 06:45:12.748143 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:45:12.748268 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260810 06:45:12.749100 18822 heartbeater.cc:347] Connected to a master server at 127.15.232.254:42065
I20260810 06:45:12.749258 18822 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:12.749485 18822 heartbeater.cc:511] Master 127.15.232.254:42065 requested a full tablet report, sending...
I20260810 06:45:12.749930 18270 ts_manager.cc:194] Registered new tserver with Master: 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757)
I20260810 06:45:12.750511 18270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:57069
I20260810 06:45:12.752522 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 18678
I20260810 06:45:12.752632 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:12.752817 16291 external_mini_cluster.cc:1428] Setting key bbb93969e430fcb7f2322f1dd028377b
I20260810 06:45:12.753664 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:45:12.756225 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.750285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:33251 (local address 127.15.232.254:42065)
0810 06:45:12.750322 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:12.750325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:12.750451 (+ 126us) server_negotiation.cc:407] Connection header received
0810 06:45:12.750523 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:12.750524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:12.750534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:12.750589 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:12.750995 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.751475 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.752185 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.752445 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.752525 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.753586 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:12.753588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:12.753588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:12.753595 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:12.755040 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.755473 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.755474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.755474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.755530 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.755748 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.755748 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.755749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.755848 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:12.755926 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:12.756093 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 06:45:12.756164 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 06:45:12.756381 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.750239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42065 (local address 127.15.232.195:33251)
0810 06:45:12.750371 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:12.750381 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:12.750466 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:12.750606 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:12.750607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:12.750821 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:12.750886 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.750888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.751573 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.751574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.752096 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.752097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.752524 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.752525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.752609 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.753124 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:12.753129 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:12.753477 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:12.755144 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:12.755145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:12.755146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:12.755384 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:12.755604 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:12.755604 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:12.755605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:12.755682 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:12.755931 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:12.755932 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:12.755985 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 06:45:12.756065 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:45:12.756184 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
Aug 10 06:45:12 dist-test-slave-vxj2 krb5kdc[18224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:45:12 dist-test-slave-vxj2 krb5kdc[18224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:12.778472 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.770863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49940 (local address 127.15.232.254:42065)
0810 06:45:12.770933 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:12.770935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:12.770947 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:45:12.771033 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:12.771034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:12.771046 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:12.771128 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:12.771799 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.772351 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.773248 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.773455 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.773539 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.776034 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:12.776035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:12.776036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:12.776043 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:12.777435 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.777772 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.777774 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.777774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.777839 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:12.778037 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:12.778038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:12.778038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:12.778154 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:12.778238 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:12.778332 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 06:45:12.778401 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:45:12.780352 18270 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:12.782058 18270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:12.790040 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.784982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:45517 (local address 127.0.0.1:57334)
0810 06:45:12.785476 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:12.785488 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:12.785604 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:12.785846 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:12.785850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:12.785853 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:12.785936 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.785938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.787567 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.787568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.788756 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.788759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.789782 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.789783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.789853 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.789865 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 06:45:12.789933 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:45:12.789975 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":80,"thread_start_us":110,"threads_started":1}
I20260810 06:45:12.790162 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.785138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.15.232.194:45517)
0810 06:45:12.785381 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:12.785385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:12.785568 (+ 183us) server_negotiation.cc:407] Connection header received
0810 06:45:12.785717 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:12.785721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:12.785750 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:12.785837 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:12.786052 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.787457 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.788865 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.789639 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.789726 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.789794 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:12.789992 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 06:45:12.790033 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":156,"threads_started":1}
I20260810 06:45:12.791036 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.784974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:33757 (local address 127.0.0.1:59588)
0810 06:45:12.785411 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:12.785462 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:12.785580 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:12.786233 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:12.786234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:12.786235 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:12.786266 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.786267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.787423 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.787425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.789105 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.789107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.790778 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.790779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.790849 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.790854 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:45:12.790921 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 06:45:12.790965 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":155,"thread_start_us":73,"threads_started":1}
I20260810 06:45:12.791114 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.785071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59588 (local address 127.15.232.195:33757)
0810 06:45:12.785256 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:12.785261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:12.785568 (+ 307us) server_negotiation.cc:407] Connection header received
0810 06:45:12.785961 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:12.785965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:12.785983 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:12.786065 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:12.786347 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.787334 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.789265 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.790103 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.790185 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.790257 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:12.790955 (+ 698us) server_negotiation.cc:299] Negotiation successful
0810 06:45:12.791002 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":109,"threads_started":1}
I20260810 06:45:12.791253 18591 tablet_service.cc:1467] Processing CreateTablet for tablet ea008e2287e4491fb6dfae3a7e0a9881 (DEFAULT_TABLE table=test-table [id=98503a3275544a37951f0903892cee4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:12.791558 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.785340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:42985 (local address 127.0.0.1:42382)
0810 06:45:12.785714 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:12.785724 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:12.785775 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:12.786064 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:12.786065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:12.786067 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:12.786130 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.786132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.787565 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.787566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.789104 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:12.789107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.791293 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.791294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:12.791355 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.791358 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:12.791436 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:45:12.791480 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":66,"thread_start_us":91,"threads_started":1}
I20260810 06:45:12.791610 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:12.785365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42382 (local address 127.15.232.193:42985)
0810 06:45:12.785587 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:12.785592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:12.785843 (+ 251us) server_negotiation.cc:407] Connection header received
0810 06:45:12.785892 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:12.785894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:12.785911 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:12.785988 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:12.786220 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.787456 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.790300 (+ 2844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:12.791094 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:12.791179 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:12.791253 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:12.791454 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 06:45:12.791495 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":89,"threads_started":1}
I20260810 06:45:12.791687 18591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea008e2287e4491fb6dfae3a7e0a9881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:12.792057 18750 tablet_service.cc:1467] Processing CreateTablet for tablet ea008e2287e4491fb6dfae3a7e0a9881 (DEFAULT_TABLE table=test-table [id=98503a3275544a37951f0903892cee4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:12.792212 18427 tablet_service.cc:1467] Processing CreateTablet for tablet ea008e2287e4491fb6dfae3a7e0a9881 (DEFAULT_TABLE table=test-table [id=98503a3275544a37951f0903892cee4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:12.792435 18750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea008e2287e4491fb6dfae3a7e0a9881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:12.792495 18427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea008e2287e4491fb6dfae3a7e0a9881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:12.795739 18853 tablet_bootstrap.cc:492] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b: Bootstrap starting.
I20260810 06:45:12.796275 18854 tablet_bootstrap.cc:492] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3: Bootstrap starting.
I20260810 06:45:12.796553 18853 tablet_bootstrap.cc:654] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:12.797047 18853 log.cc:824] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:12.797431 18854 tablet_bootstrap.cc:654] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:12.797427 18855 tablet_bootstrap.cc:492] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95: Bootstrap starting.
I20260810 06:45:12.797796 18854 log.cc:824] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:12.797926 18853 tablet_bootstrap.cc:492] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b: No bootstrap required, opened a new log
I20260810 06:45:12.798000 18853 ts_tablet_manager.cc:1397] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:45:12.798609 18854 tablet_bootstrap.cc:492] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3: No bootstrap required, opened a new log
I20260810 06:45:12.798666 18854 ts_tablet_manager.cc:1397] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:45:12.800151 18853 raft_consensus.cc:348] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } }
I20260810 06:45:12.800321 18853 raft_consensus.cc:374] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:12.800374 18853 raft_consensus.cc:729] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f598e218e9f3407983f409a6d100404b, State: Initialized, Role: FOLLOWER
I20260810 06:45:12.800506 18853 consensus_queue.cc:260] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } }
I20260810 06:45:12.800786 18853 ts_tablet_manager.cc:1428] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:12.800865 18506 heartbeater.cc:503] Master 127.15.232.254:42065 was elected leader, sending a full tablet report...
I20260810 06:45:12.800927 18854 raft_consensus.cc:348] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } }
I20260810 06:45:12.801092 18854 raft_consensus.cc:374] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:12.801138 18854 raft_consensus.cc:729] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36667d5b23cc42d2b23692f7553dcee3, State: Initialized, Role: FOLLOWER
I20260810 06:45:12.801329 18854 consensus_queue.cc:260] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } }
I20260810 06:45:12.801636 18854 ts_tablet_manager.cc:1428] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260810 06:45:12.801656 18822 heartbeater.cc:503] Master 127.15.232.254:42065 was elected leader, sending a full tablet report...
I20260810 06:45:12.801875 18855 tablet_bootstrap.cc:654] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:12.802265 18855 log.cc:824] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:12.803081 18855 tablet_bootstrap.cc:492] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95: No bootstrap required, opened a new log
I20260810 06:45:12.803146 18855 ts_tablet_manager.cc:1397] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260810 06:45:12.804685 18855 raft_consensus.cc:348] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } }
I20260810 06:45:12.804822 18855 raft_consensus.cc:374] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:12.804847 18855 raft_consensus.cc:729] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d8e1a8ac5aa48aba9a19782fe026d95, State: Initialized, Role: FOLLOWER
I20260810 06:45:12.804998 18855 consensus_queue.cc:260] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } }
I20260810 06:45:12.805312 18855 ts_tablet_manager.cc:1428] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:45:12.805369 18668 heartbeater.cc:503] Master 127.15.232.254:42065 was elected leader, sending a full tablet report...
W20260810 06:45:12.831825 18669 tablet.cc:2369] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:45:12.894313 18507 tablet.cc:2369] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:45:12.996819 18823 tablet.cc:2369] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:45:13.003444 18858 raft_consensus.cc:482] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:13.003643 18858 raft_consensus.cc:504] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } }
I20260810 06:45:13.004169 18858 leader_election.cc:288] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757), 8d8e1a8ac5aa48aba9a19782fe026d95 (127.15.232.194:45517)
I20260810 06:45:13.007297 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.004603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:58367 (local address 127.15.232.194:45517)
0810 06:45:13.004648 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.004650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.004661 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:13.004728 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.004729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.004739 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.004859 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:13.005005 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.005529 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.006580 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.006936 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.007022 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.007031 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.007189 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.007231 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:45:13.007324 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.004431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:45517 (local address 127.15.232.193:58367)
0810 06:45:13.004586 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:13.004599 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:13.004665 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:13.004863 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:13.004864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:13.004867 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:13.004930 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.004932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.005630 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.005632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.006480 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.006482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.007034 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.007035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.007104 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.007107 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:13.007175 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:45:13.007219 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":45,"thread_start_us":89,"threads_started":1}
I20260810 06:45:13.007478 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.004556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:34979 (local address 127.15.232.195:33757)
0810 06:45:13.004611 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.004613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.004666 (+ 53us) server_negotiation.cc:407] Connection header received
0810 06:45:13.004745 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.004746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.004772 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.004849 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:13.005059 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.005663 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.006681 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.007090 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.007184 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.007193 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.007365 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.007407 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 06:45:13.007493 18863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.004431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:33757 (local address 127.15.232.193:34979)
0810 06:45:13.004590 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:13.004603 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:13.004679 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:13.004855 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:13.004856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:13.004859 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:13.004958 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.004960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.005773 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.005774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.006576 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.006578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.007183 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.007185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.007279 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.007283 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:13.007359 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 06:45:13.007404 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":16,"thread_start_us":63,"threads_started":1}
I20260810 06:45:13.007864 18775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea008e2287e4491fb6dfae3a7e0a9881" candidate_uuid: "f598e218e9f3407983f409a6d100404b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36667d5b23cc42d2b23692f7553dcee3" is_pre_election: true
I20260810 06:45:13.007889 18613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea008e2287e4491fb6dfae3a7e0a9881" candidate_uuid: "f598e218e9f3407983f409a6d100404b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" is_pre_election: true
I20260810 06:45:13.008046 18613 raft_consensus.cc:2454] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f598e218e9f3407983f409a6d100404b in term 0.
I20260810 06:45:13.008045 18775 raft_consensus.cc:2454] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f598e218e9f3407983f409a6d100404b in term 0.
I20260810 06:45:13.008334 18375 leader_election.cc:302] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d8e1a8ac5aa48aba9a19782fe026d95, f598e218e9f3407983f409a6d100404b; no voters:
I20260810 06:45:13.008522 18858 raft_consensus.cc:2781] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:13.008581 18858 raft_consensus.cc:482] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:13.008611 18858 raft_consensus.cc:3037] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:13.009456 18858 raft_consensus.cc:504] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } }
I20260810 06:45:13.009611 18858 leader_election.cc:288] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [CANDIDATE]: Term 1 election: Requested vote from peers 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757), 8d8e1a8ac5aa48aba9a19782fe026d95 (127.15.232.194:45517)
I20260810 06:45:13.009768 18775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea008e2287e4491fb6dfae3a7e0a9881" candidate_uuid: "f598e218e9f3407983f409a6d100404b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36667d5b23cc42d2b23692f7553dcee3"
I20260810 06:45:13.009784 18613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea008e2287e4491fb6dfae3a7e0a9881" candidate_uuid: "f598e218e9f3407983f409a6d100404b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95"
I20260810 06:45:13.009881 18613 raft_consensus.cc:3037] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:13.009886 18775 raft_consensus.cc:3037] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:13.010632 18613 raft_consensus.cc:2454] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f598e218e9f3407983f409a6d100404b in term 1.
I20260810 06:45:13.010715 18775 raft_consensus.cc:2454] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f598e218e9f3407983f409a6d100404b in term 1.
I20260810 06:45:13.010835 18375 leader_election.cc:302] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d8e1a8ac5aa48aba9a19782fe026d95, f598e218e9f3407983f409a6d100404b; no voters:
I20260810 06:45:13.010926 18858 raft_consensus.cc:2781] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:13.011094 18858 raft_consensus.cc:686] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 1 LEADER]: Becoming Leader. State: Replica: f598e218e9f3407983f409a6d100404b, State: Running, Role: LEADER
I20260810 06:45:13.011224 18858 consensus_queue.cc:237] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } }
I20260810 06:45:13.012089 18270 catalog_manager.cc:5495] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b reported cstate change: term changed from 0 to 1, leader changed from <none> to f598e218e9f3407983f409a6d100404b (127.15.232.193). New cstate: current_term: 1 leader_uuid: "f598e218e9f3407983f409a6d100404b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } health_report { overall_health: HEALTHY } } }
Aug 10 06:45:13 dist-test-slave-vxj2 krb5kdc[18224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344312, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:13.094420 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.086144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49954 (local address 127.15.232.254:42065)
0810 06:45:13.086241 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.086243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.086546 (+ 303us) server_negotiation.cc:407] Connection header received
0810 06:45:13.086639 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.086640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.086653 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.086820 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:13.087420 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.088045 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.088884 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.089091 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.089221 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.091549 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:13.091550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:13.091551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:13.091559 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:13.093008 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:13.093515 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:13.093517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:13.093517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:13.093598 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:13.093826 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:13.093827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:13.093827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:13.093947 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:13.094022 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.094183 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.094302 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:45:13.094497 18872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.086081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42065 (local address 127.0.0.1:49954)
0810 06:45:13.086427 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:13.086439 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:13.086567 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:13.086846 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:13.086848 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:13.087250 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:13.087307 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.087310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.088149 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.088151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.088759 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.088761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.089191 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.089193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.089261 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.089856 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:13.089861 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:13.091440 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:13.093126 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:13.093136 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:13.093138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:13.093421 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:13.093675 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:13.093676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:13.093676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:13.093739 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:13.094024 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:13.094032 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:13.094084 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 06:45:13.094158 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 06:45:13.094365 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":99,"thread_start_us":199,"threads_started":1}
I20260810 06:45:13.096721 18270 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:49954:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:45:13.096985 18270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:13.098727 18427 tablet_service.cc:1467] Processing CreateTablet for tablet 0d90f9cab62d47df85eaa0639d159e1d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=119f017d5dde4b6294067f545c7ab31e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:45:13.098712 18591 tablet_service.cc:1467] Processing CreateTablet for tablet 0d90f9cab62d47df85eaa0639d159e1d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=119f017d5dde4b6294067f545c7ab31e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:45:13.098948 18591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d90f9cab62d47df85eaa0639d159e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:13.098955 18427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d90f9cab62d47df85eaa0639d159e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:13.100320 18855 tablet_bootstrap.cc:492] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: Bootstrap starting.
I20260810 06:45:13.100550 18853 tablet_bootstrap.cc:492] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b: Bootstrap starting.
I20260810 06:45:13.100705 18750 tablet_service.cc:1467] Processing CreateTablet for tablet 0d90f9cab62d47df85eaa0639d159e1d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=119f017d5dde4b6294067f545c7ab31e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:45:13.100785 18855 tablet_bootstrap.cc:654] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:13.100885 18750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d90f9cab62d47df85eaa0639d159e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:13.100991 18853 tablet_bootstrap.cc:654] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:13.102025 18853 tablet_bootstrap.cc:492] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b: No bootstrap required, opened a new log
I20260810 06:45:13.102079 18853 ts_tablet_manager.cc:1397] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 06:45:13.102242 18853 raft_consensus.cc:348] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.102319 18853 raft_consensus.cc:374] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:13.102339 18853 raft_consensus.cc:729] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f598e218e9f3407983f409a6d100404b, State: Initialized, Role: FOLLOWER
I20260810 06:45:13.102401 18853 consensus_queue.cc:260] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.102497 18853 ts_tablet_manager.cc:1428] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:13.102556 18854 tablet_bootstrap.cc:492] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3: Bootstrap starting.
I20260810 06:45:13.102594 18858 tablet_replica.cc:439] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } }
I20260810 06:45:13.102684 18858 tablet_replica.cc:442] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:13.102887 18855 tablet_bootstrap.cc:492] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: No bootstrap required, opened a new log
I20260810 06:45:13.102937 18855 ts_tablet_manager.cc:1397] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 06:45:13.103083 18855 raft_consensus.cc:348] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.103158 18855 raft_consensus.cc:374] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:13.103184 18855 raft_consensus.cc:729] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d8e1a8ac5aa48aba9a19782fe026d95, State: Initialized, Role: FOLLOWER
I20260810 06:45:13.103204 18854 tablet_bootstrap.cc:654] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:13.103237 18855 consensus_queue.cc:260] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.103360 18855 ts_tablet_manager.cc:1428] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 06:45:13.103506 18861 tablet_replica.cc:439] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } }
I20260810 06:45:13.103585 18861 tablet_replica.cc:442] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:13.104190 18854 tablet_bootstrap.cc:492] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3: No bootstrap required, opened a new log
I20260810 06:45:13.104314 18854 ts_tablet_manager.cc:1397] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260810 06:45:13.104489 18854 raft_consensus.cc:348] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.104573 18854 raft_consensus.cc:374] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:13.104600 18854 raft_consensus.cc:729] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36667d5b23cc42d2b23692f7553dcee3, State: Initialized, Role: FOLLOWER
I20260810 06:45:13.104658 18854 consensus_queue.cc:260] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.104763 18854 ts_tablet_manager.cc:1428] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:13.104877 18859 tablet_replica.cc:439] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } }
I20260810 06:45:13.104984 18859 tablet_replica.cc:442] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:13.269919 18861 raft_consensus.cc:482] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:13.270033 18861 raft_consensus.cc:504] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.270449 18861 leader_election.cc:288] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f598e218e9f3407983f409a6d100404b (127.15.232.193:42985), 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757)
I20260810 06:45:13.274694 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.270803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:33561 (local address 127.15.232.195:33757)
0810 06:45:13.270852 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.270854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.270864 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:13.270970 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.270970 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.270981 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.271057 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:13.271207 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.271941 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.273684 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.274230 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.274317 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.274325 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.274601 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.274638 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:45:13.274770 18876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.270648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:33757 (local address 127.15.232.194:33561)
0810 06:45:13.270794 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:13.270806 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:13.270904 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:13.271064 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:13.271065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:13.271068 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:13.271128 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.271129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.272046 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.272047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.273583 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.273586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.274421 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.274422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.274498 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.274504 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:45:13.274590 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:45:13.274646 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":48,"thread_start_us":78,"threads_started":1}
I20260810 06:45:13.274972 18877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.270775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:42985 (local address 127.15.232.194:54133)
0810 06:45:13.270942 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:13.270956 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:13.271045 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:13.271262 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:13.271263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:13.271266 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:13.271364 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.271366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.272232 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.272233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.273584 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.273586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.274698 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.274699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.274765 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.274769 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:13.274867 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 06:45:13.274909 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":10,"thread_start_us":69,"threads_started":1}
I20260810 06:45:13.275015 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.270820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:54133 (local address 127.15.232.193:42985)
0810 06:45:13.270904 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.270907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.271052 (+ 145us) server_negotiation.cc:407] Connection header received
0810 06:45:13.271119 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.271120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.271132 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.271196 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:13.271452 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.272114 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.273684 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.274229 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.274317 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.274325 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.274894 (+ 569us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.274946 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:45:13.275243 18775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d90f9cab62d47df85eaa0639d159e1d" candidate_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36667d5b23cc42d2b23692f7553dcee3" is_pre_election: true
I20260810 06:45:13.275355 18775 raft_consensus.cc:2454] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d8e1a8ac5aa48aba9a19782fe026d95 in term 0.
I20260810 06:45:13.275519 18451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d90f9cab62d47df85eaa0639d159e1d" candidate_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f598e218e9f3407983f409a6d100404b" is_pre_election: true
I20260810 06:45:13.275573 18538 leader_election.cc:302] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36667d5b23cc42d2b23692f7553dcee3, 8d8e1a8ac5aa48aba9a19782fe026d95; no voters:
I20260810 06:45:13.275636 18451 raft_consensus.cc:2454] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d8e1a8ac5aa48aba9a19782fe026d95 in term 0.
I20260810 06:45:13.275722 18861 raft_consensus.cc:2781] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:13.275784 18861 raft_consensus.cc:482] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:13.275808 18861 raft_consensus.cc:3037] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:13.276543 18861 raft_consensus.cc:504] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.276718 18861 leader_election.cc:288] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [CANDIDATE]: Term 1 election: Requested vote from peers f598e218e9f3407983f409a6d100404b (127.15.232.193:42985), 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757)
I20260810 06:45:13.276852 18451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d90f9cab62d47df85eaa0639d159e1d" candidate_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f598e218e9f3407983f409a6d100404b"
I20260810 06:45:13.276885 18775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d90f9cab62d47df85eaa0639d159e1d" candidate_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36667d5b23cc42d2b23692f7553dcee3"
I20260810 06:45:13.276950 18451 raft_consensus.cc:3037] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:13.276973 18775 raft_consensus.cc:3037] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:13.277619 18775 raft_consensus.cc:2454] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d8e1a8ac5aa48aba9a19782fe026d95 in term 1.
I20260810 06:45:13.277745 18451 raft_consensus.cc:2454] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d8e1a8ac5aa48aba9a19782fe026d95 in term 1.
I20260810 06:45:13.277845 18538 leader_election.cc:302] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36667d5b23cc42d2b23692f7553dcee3, 8d8e1a8ac5aa48aba9a19782fe026d95; no voters:
I20260810 06:45:13.277937 18861 raft_consensus.cc:2781] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:13.278074 18861 raft_consensus.cc:686] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 LEADER]: Becoming Leader. State: Replica: 8d8e1a8ac5aa48aba9a19782fe026d95, State: Running, Role: LEADER
I20260810 06:45:13.278168 18861 consensus_queue.cc:237] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.278640 18878 tablet_replica.cc:439] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: TxnStatusTablet state changed. Reason: New leader 8d8e1a8ac5aa48aba9a19782fe026d95. Latest consensus state: current_term: 1 leader_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } }
I20260810 06:45:13.278738 18878 tablet_replica.cc:442] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:45:13.279017 18271 catalog_manager.cc:5495] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d8e1a8ac5aa48aba9a19782fe026d95 (127.15.232.194). New cstate: current_term: 1 leader_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } health_report { overall_health: UNKNOWN } } }
I20260810 06:45:13.279363 18880 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:45:13.279495 18880 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:45:13.286912 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.283883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57346 (local address 127.15.232.194:45517)
0810 06:45:13.283924 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.283926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.283996 (+ 70us) server_negotiation.cc:407] Connection header received
0810 06:45:13.284085 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.284086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.284100 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.284157 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:13.284342 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.284893 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.286051 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.286280 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.286356 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.286605 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:13.286700 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:13.286758 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.286836 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.286856 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:45:13.286974 18872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.283835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:45517 (local address 127.0.0.1:57346)
0810 06:45:13.283928 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:13.283942 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:13.284007 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:13.284181 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:13.284182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:13.284183 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:13.284239 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.284243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.284981 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.284982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.285946 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.285950 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.286389 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.286390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.286482 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.286503 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:13.286766 (+ 263us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:13.286775 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:45:13.286830 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 06:45:13.286892 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 06:45:13.289143 18775 raft_consensus.cc:1261] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Refusing update from remote peer 8d8e1a8ac5aa48aba9a19782fe026d95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:13.289244 18451 raft_consensus.cc:1261] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Refusing update from remote peer 8d8e1a8ac5aa48aba9a19782fe026d95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:13.289556 18878 consensus_queue.cc:1035] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:13.289706 18861 consensus_queue.cc:1035] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:13.290892 18859 tablet_replica.cc:439] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3: TxnStatusTablet state changed. Reason: New leader 8d8e1a8ac5aa48aba9a19782fe026d95. Latest consensus state: current_term: 1 leader_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } }
I20260810 06:45:13.290983 18859 tablet_replica.cc:442] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:13.291008 18858 tablet_replica.cc:439] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b: TxnStatusTablet state changed. Reason: New leader 8d8e1a8ac5aa48aba9a19782fe026d95. Latest consensus state: current_term: 1 leader_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } }
I20260810 06:45:13.291123 18858 tablet_replica.cc:442] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:13.294200 18861 tablet_replica.cc:439] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } }
I20260810 06:45:13.294292 18861 tablet_replica.cc:442] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:45:13.294420 18878 tablet_replica.cc:439] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } }
I20260810 06:45:13.294515 18878 tablet_replica.cc:442] T 0d90f9cab62d47df85eaa0639d159e1d P 8d8e1a8ac5aa48aba9a19782fe026d95: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:45:13.294878 18859 tablet_replica.cc:439] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } }
I20260810 06:45:13.294960 18859 tablet_replica.cc:442] T 0d90f9cab62d47df85eaa0639d159e1d P 36667d5b23cc42d2b23692f7553dcee3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:13.295253 18858 tablet_replica.cc:439] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } } }
I20260810 06:45:13.295358 18858 tablet_replica.cc:442] T 0d90f9cab62d47df85eaa0639d159e1d P f598e218e9f3407983f409a6d100404b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:13.295360 18884 mvcc.cc:205] Tried to move back new op lower bound from 7316866307229257728 to 7316866307187806208. Current Snapshot: MvccSnapshot[applied={T|T < 7316866307229257728}]
I20260810 06:45:13.300626 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.297553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42392 (local address 127.15.232.193:42985)
0810 06:45:13.297597 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.297599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.297711 (+ 112us) server_negotiation.cc:407] Connection header received
0810 06:45:13.297884 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.297886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.297907 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.297982 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:13.298157 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.298793 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.299671 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.299908 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.299994 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.300194 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:13.300346 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:13.300447 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.300515 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.300540 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:45:13.306237 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.303361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:33649 (local address 127.15.232.194:45517)
0810 06:45:13.303422 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.303424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.303552 (+ 128us) server_negotiation.cc:407] Connection header received
0810 06:45:13.303613 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.303614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.303629 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.303675 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:13.303808 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.304304 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.305470 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.305682 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.305814 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.306009 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:13.306038 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:13.306082 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.306157 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.306176 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:45:13.306299 18895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.303308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:45517 (local address 127.15.232.193:33649)
0810 06:45:13.303483 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:13.303493 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:13.303554 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:13.303689 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:13.303690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:13.303692 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:13.303748 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.303749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.304390 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.304391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.305367 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.305370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.305816 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.305817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.305891 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.305907 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:13.306101 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:13.306103 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:45:13.306154 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 06:45:13.306199 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260810 06:45:13.309099 18775 raft_consensus.cc:1261] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Refusing update from remote peer f598e218e9f3407983f409a6d100404b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:13.309180 18613 raft_consensus.cc:1261] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Refusing update from remote peer f598e218e9f3407983f409a6d100404b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:13.309425 18864 consensus_queue.cc:1035] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:13.309552 18858 consensus_queue.cc:1035] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:13.311360 18883 mvcc.cc:205] Tried to move back new op lower bound from 7316866307312005120 to 7316866306094706688. Current Snapshot: MvccSnapshot[applied={T|T < 7316866307312005120}]
I20260810 06:45:13.319674 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.316260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:46061 (local address 127.15.232.193:42985)
0810 06:45:13.316306 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.316308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.316465 (+ 157us) server_negotiation.cc:407] Connection header received
0810 06:45:13.316513 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.316514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.316525 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.316594 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:13.316773 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.317555 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.318789 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.319099 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.319171 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.319371 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:13.319406 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:13.319462 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.319574 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.319598 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:45:13.319680 18904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.316215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:42985 (local address 127.15.232.194:46061)
0810 06:45:13.316368 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:13.316380 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:13.316457 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:13.316610 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:13.316611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:13.316613 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:13.316688 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.316690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.317655 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.317656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.318700 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.318703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.319202 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.319202 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.319268 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.319284 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:13.319500 (+ 216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:13.319503 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:13.319558 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 06:45:13.319600 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260810 06:45:13.329329 18271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:13.329449 18271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:13.330258 18271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256: Sending DeleteTablet for 3 replicas of tablet ea008e2287e4491fb6dfae3a7e0a9881
I20260810 06:45:13.330581 18750 tablet_service.cc:1514] Processing DeleteTablet for tablet ea008e2287e4491fb6dfae3a7e0a9881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:13 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:59588
I20260810 06:45:13.330649 18427 tablet_service.cc:1514] Processing DeleteTablet for tablet ea008e2287e4491fb6dfae3a7e0a9881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:13 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:42382
I20260810 06:45:13.330950 18907 tablet_replica.cc:330] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3: stopping tablet replica
I20260810 06:45:13.330960 18591 tablet_service.cc:1514] Processing DeleteTablet for tablet ea008e2287e4491fb6dfae3a7e0a9881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:13 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:57334
I20260810 06:45:13.331140 18907 raft_consensus.cc:2229] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:13.331254 18907 raft_consensus.cc:2258] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:13.331372 18271 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:13.331509 18271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:13.331930 18907 ts_tablet_manager.cc:1905] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:13.332528 18750 tablet_service.cc:1467] Processing CreateTablet for tablet 7bd0540fcf69459c9103fcb0cd7cefd4 (DEFAULT_TABLE table=test-table [id=aa73ed87ba884a6cab0c723799e91e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:13.332634 18750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bd0540fcf69459c9103fcb0cd7cefd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:13.332693 18427 tablet_service.cc:1467] Processing CreateTablet for tablet 7bd0540fcf69459c9103fcb0cd7cefd4 (DEFAULT_TABLE table=test-table [id=aa73ed87ba884a6cab0c723799e91e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:13.332808 18427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bd0540fcf69459c9103fcb0cd7cefd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:13.334024 18854 tablet_bootstrap.cc:492] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3: Bootstrap starting.
I20260810 06:45:13.334159 18853 tablet_bootstrap.cc:492] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b: Bootstrap starting.
I20260810 06:45:13.334416 18591 tablet_service.cc:1467] Processing CreateTablet for tablet 7bd0540fcf69459c9103fcb0cd7cefd4 (DEFAULT_TABLE table=test-table [id=aa73ed87ba884a6cab0c723799e91e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:13.334441 18854 tablet_bootstrap.cc:654] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:13.334525 18591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bd0540fcf69459c9103fcb0cd7cefd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:13.334604 18853 tablet_bootstrap.cc:654] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:13.335978 18855 tablet_bootstrap.cc:492] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95: Bootstrap starting.
I20260810 06:45:13.336585 18855 tablet_bootstrap.cc:654] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:13.336722 18907 ts_tablet_manager.cc:1918] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:13.336843 18907 log.cc:1196] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/ea008e2287e4491fb6dfae3a7e0a9881
I20260810 06:45:13.337214 18907 ts_tablet_manager.cc:1939] T ea008e2287e4491fb6dfae3a7e0a9881 P 36667d5b23cc42d2b23692f7553dcee3: Deleting consensus metadata
I20260810 06:45:13.337716 18855 tablet_bootstrap.cc:492] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95: No bootstrap required, opened a new log
I20260810 06:45:13.337705 18256 catalog_manager.cc:4841] TS 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757): tablet ea008e2287e4491fb6dfae3a7e0a9881 (table test-table [id=98503a3275544a37951f0903892cee4f]) successfully deleted
I20260810 06:45:13.337772 18855 ts_tablet_manager.cc:1397] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 06:45:13.337921 18855 raft_consensus.cc:348] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.338001 18855 raft_consensus.cc:374] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:13.338023 18855 raft_consensus.cc:729] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d8e1a8ac5aa48aba9a19782fe026d95, State: Initialized, Role: FOLLOWER
I20260810 06:45:13.338064 18855 consensus_queue.cc:260] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.338167 18855 ts_tablet_manager.cc:1428] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 06:45:13.338407 18908 tablet_replica.cc:330] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b: stopping tablet replica
I20260810 06:45:13.338514 18908 raft_consensus.cc:2229] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:13.338613 18908 raft_consensus.cc:2258] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:13.338891 18909 tablet_replica.cc:330] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95: stopping tablet replica
I20260810 06:45:13.338997 18909 raft_consensus.cc:2229] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:13.339056 18909 raft_consensus.cc:2258] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:13.339392 18909 ts_tablet_manager.cc:1905] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:13.341023 18909 ts_tablet_manager.cc:1918] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:13.341100 18909 log.cc:1196] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/ea008e2287e4491fb6dfae3a7e0a9881
I20260810 06:45:13.341405 18909 ts_tablet_manager.cc:1939] T ea008e2287e4491fb6dfae3a7e0a9881 P 8d8e1a8ac5aa48aba9a19782fe026d95: Deleting consensus metadata
I20260810 06:45:13.341674 18854 tablet_bootstrap.cc:492] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3: No bootstrap required, opened a new log
I20260810 06:45:13.341754 18854 ts_tablet_manager.cc:1397] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20260810 06:45:13.341805 18853 tablet_bootstrap.cc:492] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b: No bootstrap required, opened a new log
I20260810 06:45:13.341849 18853 ts_tablet_manager.cc:1397] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260810 06:45:13.341862 18908 ts_tablet_manager.cc:1905] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:13.341964 18854 raft_consensus.cc:348] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.342047 18854 raft_consensus.cc:374] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:13.342012 18853 raft_consensus.cc:348] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.342077 18854 raft_consensus.cc:729] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36667d5b23cc42d2b23692f7553dcee3, State: Initialized, Role: FOLLOWER
I20260810 06:45:13.342097 18853 raft_consensus.cc:374] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:13.342128 18853 raft_consensus.cc:729] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f598e218e9f3407983f409a6d100404b, State: Initialized, Role: FOLLOWER
I20260810 06:45:13.342123 18854 consensus_queue.cc:260] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.342171 18853 consensus_queue.cc:260] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.342224 18854 ts_tablet_manager.cc:1428] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:13.342254 18853 ts_tablet_manager.cc:1428] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 06:45:13.342541 18254 catalog_manager.cc:4841] TS 8d8e1a8ac5aa48aba9a19782fe026d95 (127.15.232.194:45517): tablet ea008e2287e4491fb6dfae3a7e0a9881 (table test-table [id=98503a3275544a37951f0903892cee4f]) successfully deleted
I20260810 06:45:13.343817 18908 ts_tablet_manager.cc:1918] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:13.343900 18908 log.cc:1196] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/wal/wals/ea008e2287e4491fb6dfae3a7e0a9881
I20260810 06:45:13.344162 18908 ts_tablet_manager.cc:1939] T ea008e2287e4491fb6dfae3a7e0a9881 P f598e218e9f3407983f409a6d100404b: Deleting consensus metadata
I20260810 06:45:13.344558 18254 catalog_manager.cc:4841] TS f598e218e9f3407983f409a6d100404b (127.15.232.193:42985): tablet ea008e2287e4491fb6dfae3a7e0a9881 (table test-table [id=98503a3275544a37951f0903892cee4f]) successfully deleted
I20260810 06:45:13.373436 18859 raft_consensus.cc:482] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:13.373556 18859 raft_consensus.cc:504] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.373908 18859 leader_election.cc:288] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f598e218e9f3407983f409a6d100404b (127.15.232.193:42985), 8d8e1a8ac5aa48aba9a19782fe026d95 (127.15.232.194:45517)
I20260810 06:45:13.378154 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.374208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:45763 (local address 127.15.232.193:42985)
0810 06:45:13.374259 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.374262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.374513 (+ 251us) server_negotiation.cc:407] Connection header received
0810 06:45:13.374629 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.374630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.374643 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.374674 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:13.374920 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.375644 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.377172 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.377714 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.377771 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.377781 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.378033 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.378075 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:45:13.378165 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.374498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:55825 (local address 127.15.232.194:45517)
0810 06:45:13.374542 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.374545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.374652 (+ 107us) server_negotiation.cc:407] Connection header received
0810 06:45:13.374752 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.374752 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.374763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.374793 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:13.374997 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.375719 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.377210 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.377774 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.377823 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.377830 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.378054 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.378089 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:45:13.378443 18916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.374330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:45517 (local address 127.15.232.195:55825)
0810 06:45:13.374595 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:13.374605 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:13.374677 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:13.374856 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:13.374857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:13.374859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:13.374926 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.374927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.375823 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.375824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.377065 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.377068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.377891 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.377892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.377973 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.377977 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:13.378307 (+ 330us) client_negotiation.cc:240] Negotiation successful
0810 06:45:13.378351 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":24,"thread_start_us":74,"threads_started":1}
I20260810 06:45:13.378656 18613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bd0540fcf69459c9103fcb0cd7cefd4" candidate_uuid: "36667d5b23cc42d2b23692f7553dcee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" is_pre_election: true
I20260810 06:45:13.378747 18613 raft_consensus.cc:2454] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36667d5b23cc42d2b23692f7553dcee3 in term 0.
I20260810 06:45:13.378934 18695 leader_election.cc:302] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36667d5b23cc42d2b23692f7553dcee3, 8d8e1a8ac5aa48aba9a19782fe026d95; no voters:
I20260810 06:45:13.379001 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.374120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:42985 (local address 127.15.232.195:45763)
0810 06:45:13.374428 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:13.374441 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:13.374544 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:13.374747 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:13.374747 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:13.374749 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:13.374829 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.374831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.375761 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.375762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.377030 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.377034 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.377851 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.377852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.377935 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.377939 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:13.378835 (+ 896us) client_negotiation.cc:240] Negotiation successful
0810 06:45:13.378883 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":5,"thread_start_us":100,"threads_started":1}
I20260810 06:45:13.379081 18859 raft_consensus.cc:2781] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:13.379347 18859 raft_consensus.cc:482] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:13.379379 18859 raft_consensus.cc:3037] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:13.379565 18451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bd0540fcf69459c9103fcb0cd7cefd4" candidate_uuid: "36667d5b23cc42d2b23692f7553dcee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f598e218e9f3407983f409a6d100404b" is_pre_election: true
I20260810 06:45:13.379655 18451 raft_consensus.cc:2454] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36667d5b23cc42d2b23692f7553dcee3 in term 0.
I20260810 06:45:13.380317 18859 raft_consensus.cc:504] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.380470 18859 leader_election.cc:288] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [CANDIDATE]: Term 1 election: Requested vote from peers f598e218e9f3407983f409a6d100404b (127.15.232.193:42985), 8d8e1a8ac5aa48aba9a19782fe026d95 (127.15.232.194:45517)
I20260810 06:45:13.380712 18613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bd0540fcf69459c9103fcb0cd7cefd4" candidate_uuid: "36667d5b23cc42d2b23692f7553dcee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95"
I20260810 06:45:13.380782 18613 raft_consensus.cc:3037] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:13.381601 18613 raft_consensus.cc:2454] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36667d5b23cc42d2b23692f7553dcee3 in term 1.
I20260810 06:45:13.381764 18695 leader_election.cc:302] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36667d5b23cc42d2b23692f7553dcee3, 8d8e1a8ac5aa48aba9a19782fe026d95; no voters:
I20260810 06:45:13.381842 18859 raft_consensus.cc:2781] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:13.381999 18859 raft_consensus.cc:686] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 LEADER]: Becoming Leader. State: Replica: 36667d5b23cc42d2b23692f7553dcee3, State: Running, Role: LEADER
I20260810 06:45:13.382063 18859 consensus_queue.cc:237] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:13.382794 18271 catalog_manager.cc:5495] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195). New cstate: current_term: 1 leader_uuid: "36667d5b23cc42d2b23692f7553dcee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } health_report { overall_health: HEALTHY } } }
I20260810 06:45:13.384249 18451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bd0540fcf69459c9103fcb0cd7cefd4" candidate_uuid: "36667d5b23cc42d2b23692f7553dcee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f598e218e9f3407983f409a6d100404b"
I20260810 06:45:13.384399 18451 raft_consensus.cc:3037] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:13.385608 18451 raft_consensus.cc:2454] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36667d5b23cc42d2b23692f7553dcee3 in term 1.
I20260810 06:45:13.402930 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.399554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59604 (local address 127.15.232.195:33757)
0810 06:45:13.399596 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.399598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.399605 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:45:13.399652 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.399653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.399681 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.399757 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:13.399940 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.400679 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.401826 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.402117 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.402178 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.402394 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:13.402526 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:13.402589 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.402827 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.402851 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:45:13.409305 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.406148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:60001 (local address 127.15.232.194:45517)
0810 06:45:13.406188 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.406190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.406199 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:13.406276 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.406277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.406289 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.406353 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:13.406538 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.407251 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.408429 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.408716 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.408795 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.408999 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:13.409032 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:13.409092 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.409187 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.409210 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:45:13.409364 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.405952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:45517 (local address 127.15.232.195:60001)
0810 06:45:13.406140 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:13.406153 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:13.406209 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:13.406370 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:13.406371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:13.406373 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:13.406445 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.406447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.407360 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.407361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.408315 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.408319 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.408806 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.408807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.408881 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.408898 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:13.409108 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:13.409110 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:45:13.409184 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 06:45:13.409234 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260810 06:45:13.412290 18613 raft_consensus.cc:1261] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Refusing update from remote peer 36667d5b23cc42d2b23692f7553dcee3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:13.412293 18451 raft_consensus.cc:1261] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Refusing update from remote peer 36667d5b23cc42d2b23692f7553dcee3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:13.412565 18859 consensus_queue.cc:1035] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:13.412832 18859 consensus_queue.cc:1035] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:13.415392 18885 mvcc.cc:205] Tried to move back new op lower bound from 7316866307734380544 to 7316866307613503488. Current Snapshot: MvccSnapshot[applied={T|T < 7316866307734380544}]
I20260810 06:45:13.422778 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.419653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:55567 (local address 127.15.232.195:33757)
0810 06:45:13.419692 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:13.419694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:13.419702 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:13.419729 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:13.419729 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:13.419740 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:13.419794 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:13.419972 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.420692 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.421935 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.422208 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.422283 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.422470 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:13.422515 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:13.422596 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:13.422663 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 06:45:13.422692 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:45:13.422801 18904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:13.419556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:33757 (local address 127.15.232.194:55567)
0810 06:45:13.419613 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:13.419627 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:13.419676 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:13.419845 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:13.419846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:13.419848 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:13.419895 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.419897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.420825 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.420827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.421819 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:13.421823 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:13.422331 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:13.422333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:13.422387 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:13.422394 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:13.422605 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:13.422608 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:13.422658 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 06:45:13.422709 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 06:45:13.423517 18901 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 06:45:14.949674 18270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:14.949890 18270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:14.950875 18270 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256: Sending DeleteTablet for 3 replicas of tablet 7bd0540fcf69459c9103fcb0cd7cefd4
I20260810 06:45:14.951169 18750 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bd0540fcf69459c9103fcb0cd7cefd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:14 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:59588
I20260810 06:45:14.951236 18427 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bd0540fcf69459c9103fcb0cd7cefd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:14 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:42382
I20260810 06:45:14.951239 18591 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bd0540fcf69459c9103fcb0cd7cefd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:14 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:57334
I20260810 06:45:14.951443 18953 tablet_replica.cc:330] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3: stopping tablet replica
I20260810 06:45:14.951514 18953 raft_consensus.cc:2229] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:14.951570 18954 tablet_replica.cc:330] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b: stopping tablet replica
I20260810 06:45:14.951615 18953 raft_consensus.cc:2258] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:14.951639 18954 raft_consensus.cc:2229] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:14.951694 18954 raft_consensus.cc:2258] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:14.951987 18954 ts_tablet_manager.cc:1905] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:14.954174 18954 ts_tablet_manager.cc:1918] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:45:14.954262 18954 log.cc:1196] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/wal/wals/7bd0540fcf69459c9103fcb0cd7cefd4
I20260810 06:45:14.954524 18954 ts_tablet_manager.cc:1939] T 7bd0540fcf69459c9103fcb0cd7cefd4 P f598e218e9f3407983f409a6d100404b: Deleting consensus metadata
I20260810 06:45:14.954725 18953 ts_tablet_manager.cc:1905] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:14.955161 18254 catalog_manager.cc:4841] TS f598e218e9f3407983f409a6d100404b (127.15.232.193:42985): tablet 7bd0540fcf69459c9103fcb0cd7cefd4 (table test-table [id=aa73ed87ba884a6cab0c723799e91e2f]) successfully deleted
I20260810 06:45:14.956400 18953 ts_tablet_manager.cc:1918] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:45:14.956453 18953 log.cc:1196] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/7bd0540fcf69459c9103fcb0cd7cefd4
I20260810 06:45:14.956698 18953 ts_tablet_manager.cc:1939] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 36667d5b23cc42d2b23692f7553dcee3: Deleting consensus metadata
I20260810 06:45:14.957329 18955 tablet_replica.cc:330] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95: stopping tablet replica
I20260810 06:45:14.957408 18955 raft_consensus.cc:2229] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:14.957465 18955 raft_consensus.cc:2258] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:14.957476 18256 catalog_manager.cc:4841] TS 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757): tablet 7bd0540fcf69459c9103fcb0cd7cefd4 (table test-table [id=aa73ed87ba884a6cab0c723799e91e2f]) successfully deleted
I20260810 06:45:14.957764 18955 ts_tablet_manager.cc:1905] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:14.960108 18955 ts_tablet_manager.cc:1918] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:45:14.960162 18955 log.cc:1196] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/7bd0540fcf69459c9103fcb0cd7cefd4
I20260810 06:45:14.960405 18955 ts_tablet_manager.cc:1939] T 7bd0540fcf69459c9103fcb0cd7cefd4 P 8d8e1a8ac5aa48aba9a19782fe026d95: Deleting consensus metadata
I20260810 06:45:14.960767 18254 catalog_manager.cc:4841] TS 8d8e1a8ac5aa48aba9a19782fe026d95 (127.15.232.194:45517): tablet 7bd0540fcf69459c9103fcb0cd7cefd4 (table test-table [id=aa73ed87ba884a6cab0c723799e91e2f]) successfully deleted
Aug 10 06:45:14 dist-test-slave-vxj2 krb5kdc[18224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344314, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:45:14 dist-test-slave-vxj2 krb5kdc[18224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344314, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:14.982396 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:14.969671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58254 (local address 127.15.232.254:42065)
0810 06:45:14.969811 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:14.969813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:14.969892 (+ 79us) server_negotiation.cc:407] Connection header received
0810 06:45:14.969934 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:14.969934 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:14.969945 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:14.970008 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:14.970646 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:14.971185 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:14.972054 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:14.972252 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:14.972308 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:14.979432 (+ 7124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:14.979433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:14.979434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:14.979449 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:14.981202 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:14.981564 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:14.981565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:14.981565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:14.981630 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:14.981845 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:14.981846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:14.981846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:14.981970 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:14.982083 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:14.982195 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 06:45:14.982285 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260810 06:45:14.983923 18270 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:14.984073 18270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:14.985690 18591 tablet_service.cc:1467] Processing CreateTablet for tablet cccf774ee5d9487cbe5c22e8de6b7196 (DEFAULT_TABLE table=test-table [id=7abdde8818ba42e3a1ebafd0c5019ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:14.985692 18750 tablet_service.cc:1467] Processing CreateTablet for tablet cccf774ee5d9487cbe5c22e8de6b7196 (DEFAULT_TABLE table=test-table [id=7abdde8818ba42e3a1ebafd0c5019ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:14.985857 18591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cccf774ee5d9487cbe5c22e8de6b7196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:14.985875 18750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cccf774ee5d9487cbe5c22e8de6b7196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:14.987226 18964 tablet_bootstrap.cc:492] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95: Bootstrap starting.
I20260810 06:45:14.987295 18427 tablet_service.cc:1467] Processing CreateTablet for tablet cccf774ee5d9487cbe5c22e8de6b7196 (DEFAULT_TABLE table=test-table [id=7abdde8818ba42e3a1ebafd0c5019ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:14.987437 18427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cccf774ee5d9487cbe5c22e8de6b7196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:14.987457 18965 tablet_bootstrap.cc:492] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3: Bootstrap starting.
I20260810 06:45:14.987625 18964 tablet_bootstrap.cc:654] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:14.988005 18965 tablet_bootstrap.cc:654] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:14.988384 18964 tablet_bootstrap.cc:492] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95: No bootstrap required, opened a new log
I20260810 06:45:14.988444 18964 ts_tablet_manager.cc:1397] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:14.988551 18964 raft_consensus.cc:348] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:14.988626 18964 raft_consensus.cc:374] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:14.988646 18964 raft_consensus.cc:729] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d8e1a8ac5aa48aba9a19782fe026d95, State: Initialized, Role: FOLLOWER
I20260810 06:45:14.988714 18964 consensus_queue.cc:260] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:14.988742 18965 tablet_bootstrap.cc:492] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3: No bootstrap required, opened a new log
I20260810 06:45:14.988845 18964 ts_tablet_manager.cc:1428] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:14.988868 18965 ts_tablet_manager.cc:1397] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:14.989006 18965 raft_consensus.cc:348] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:14.989087 18965 raft_consensus.cc:374] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:14.989113 18965 raft_consensus.cc:729] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36667d5b23cc42d2b23692f7553dcee3, State: Initialized, Role: FOLLOWER
I20260810 06:45:14.989245 18965 consensus_queue.cc:260] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:14.989323 18969 tablet_bootstrap.cc:492] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b: Bootstrap starting.
I20260810 06:45:14.989439 18965 ts_tablet_manager.cc:1428] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 06:45:14.989737 18969 tablet_bootstrap.cc:654] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:14.990351 18969 tablet_bootstrap.cc:492] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b: No bootstrap required, opened a new log
I20260810 06:45:14.990415 18969 ts_tablet_manager.cc:1397] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:14.990553 18969 raft_consensus.cc:348] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:14.990629 18969 raft_consensus.cc:374] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:14.990661 18969 raft_consensus.cc:729] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f598e218e9f3407983f409a6d100404b, State: Initialized, Role: FOLLOWER
I20260810 06:45:14.990716 18969 consensus_queue.cc:260] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:14.990885 18969 ts_tablet_manager.cc:1428] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:15.210686 18936 raft_consensus.cc:482] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:15.210814 18936 raft_consensus.cc:504] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:15.211045 18936 leader_election.cc:288] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f598e218e9f3407983f409a6d100404b (127.15.232.193:42985), 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757)
I20260810 06:45:15.211292 18775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cccf774ee5d9487cbe5c22e8de6b7196" candidate_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36667d5b23cc42d2b23692f7553dcee3" is_pre_election: true
I20260810 06:45:15.211385 18775 raft_consensus.cc:2454] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d8e1a8ac5aa48aba9a19782fe026d95 in term 0.
I20260810 06:45:15.211344 18451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cccf774ee5d9487cbe5c22e8de6b7196" candidate_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f598e218e9f3407983f409a6d100404b" is_pre_election: true
I20260810 06:45:15.211479 18451 raft_consensus.cc:2454] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d8e1a8ac5aa48aba9a19782fe026d95 in term 0.
I20260810 06:45:15.211565 18538 leader_election.cc:302] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36667d5b23cc42d2b23692f7553dcee3, 8d8e1a8ac5aa48aba9a19782fe026d95; no voters:
I20260810 06:45:15.211697 18936 raft_consensus.cc:2781] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:15.211736 18936 raft_consensus.cc:482] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:15.211760 18936 raft_consensus.cc:3037] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:15.212466 18936 raft_consensus.cc:504] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:15.212599 18936 leader_election.cc:288] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [CANDIDATE]: Term 1 election: Requested vote from peers f598e218e9f3407983f409a6d100404b (127.15.232.193:42985), 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757)
I20260810 06:45:15.212765 18451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cccf774ee5d9487cbe5c22e8de6b7196" candidate_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f598e218e9f3407983f409a6d100404b"
I20260810 06:45:15.212797 18775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cccf774ee5d9487cbe5c22e8de6b7196" candidate_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36667d5b23cc42d2b23692f7553dcee3"
I20260810 06:45:15.212867 18451 raft_consensus.cc:3037] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:15.212886 18775 raft_consensus.cc:3037] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:15.213759 18451 raft_consensus.cc:2454] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d8e1a8ac5aa48aba9a19782fe026d95 in term 1.
I20260810 06:45:15.213905 18775 raft_consensus.cc:2454] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d8e1a8ac5aa48aba9a19782fe026d95 in term 1.
I20260810 06:45:15.214005 18536 leader_election.cc:302] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d8e1a8ac5aa48aba9a19782fe026d95, f598e218e9f3407983f409a6d100404b; no voters:
I20260810 06:45:15.214109 18936 raft_consensus.cc:2781] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:15.214193 18936 raft_consensus.cc:686] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 LEADER]: Becoming Leader. State: Replica: 8d8e1a8ac5aa48aba9a19782fe026d95, State: Running, Role: LEADER
I20260810 06:45:15.214262 18936 consensus_queue.cc:237] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:15.214955 18270 catalog_manager.cc:5495] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d8e1a8ac5aa48aba9a19782fe026d95 (127.15.232.194). New cstate: current_term: 1 leader_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } health_report { overall_health: UNKNOWN } } }
I20260810 06:45:15.277200 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:15.274319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49430 (local address 127.15.232.194:45517)
0810 06:45:15.274492 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:15.274494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:15.274503 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:15.274534 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:15.274535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:15.274545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:15.274636 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:15.274881 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:15.275442 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:15.276319 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:15.276560 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:15.276648 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:15.276844 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:15.276890 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:15.276960 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:15.277023 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 06:45:15.277050 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":112,"threads_started":1}
I20260810 06:45:15.282730 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:15.280063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:49703 (local address 127.15.232.194:45517)
0810 06:45:15.280128 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:15.280130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:15.280140 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:15.280171 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:15.280173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:15.280183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:15.280233 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:15.280402 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:15.281008 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:15.281969 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:15.282181 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:15.282256 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:15.282445 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:15.282472 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:15.282521 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:15.282598 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 06:45:15.282625 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:45:15.282740 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:15.279843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:45517 (local address 127.15.232.194:49703)
0810 06:45:15.280024 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:15.280037 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:15.280104 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:15.280268 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:15.280269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:15.280271 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:15.280314 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:15.280316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:15.281122 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:15.281123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:15.281867 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:15.281870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:15.282298 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:15.282299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:15.282350 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:15.282355 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:15.282542 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:15.282544 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:45:15.282584 (+ 40us) client_negotiation.cc:240] Negotiation successful
0810 06:45:15.282625 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
I20260810 06:45:15.285444 18451 raft_consensus.cc:1261] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Refusing update from remote peer 8d8e1a8ac5aa48aba9a19782fe026d95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:15.285434 18775 raft_consensus.cc:1261] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Refusing update from remote peer 8d8e1a8ac5aa48aba9a19782fe026d95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:15.285696 18930 consensus_queue.cc:1035] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:15.285840 18861 consensus_queue.cc:1035] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:15.297048 18271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:15.297194 18271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:15.297945 18271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256: Sending DeleteTablet for 3 replicas of tablet cccf774ee5d9487cbe5c22e8de6b7196
I20260810 06:45:15.298252 18427 tablet_service.cc:1514] Processing DeleteTablet for tablet cccf774ee5d9487cbe5c22e8de6b7196 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:15 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:42382
I20260810 06:45:15.298239 18750 tablet_service.cc:1514] Processing DeleteTablet for tablet cccf774ee5d9487cbe5c22e8de6b7196 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:15 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:59588
I20260810 06:45:15.298322 18590 tablet_service.cc:1514] Processing DeleteTablet for tablet cccf774ee5d9487cbe5c22e8de6b7196 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:15 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:57334
I20260810 06:45:15.298352 18954 tablet_replica.cc:330] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b: stopping tablet replica
I20260810 06:45:15.298357 18953 tablet_replica.cc:330] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3: stopping tablet replica
I20260810 06:45:15.298424 18954 raft_consensus.cc:2229] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:15.298421 18953 raft_consensus.cc:2229] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:15.298424 18955 tablet_replica.cc:330] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95: stopping tablet replica
I20260810 06:45:15.298468 18954 raft_consensus.cc:2258] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:15.298470 18953 raft_consensus.cc:2258] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:15.298497 18955 raft_consensus.cc:2229] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:15.298594 18955 raft_consensus.cc:2258] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:15.298862 18271 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 06:45:15.299007 18953 ts_tablet_manager.cc:1905] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 06:45:15.299031 18271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:15.299099 18955 ts_tablet_manager.cc:1905] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:15.300602 18955 ts_tablet_manager.cc:1918] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:15.300662 18955 log.cc:1196] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/cccf774ee5d9487cbe5c22e8de6b7196
I20260810 06:45:15.300705 18953 ts_tablet_manager.cc:1918] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:15.300783 18953 log.cc:1196] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/cccf774ee5d9487cbe5c22e8de6b7196
I20260810 06:45:15.300896 18955 ts_tablet_manager.cc:1939] T cccf774ee5d9487cbe5c22e8de6b7196 P 8d8e1a8ac5aa48aba9a19782fe026d95: Deleting consensus metadata
I20260810 06:45:15.301038 18953 ts_tablet_manager.cc:1939] T cccf774ee5d9487cbe5c22e8de6b7196 P 36667d5b23cc42d2b23692f7553dcee3: Deleting consensus metadata
I20260810 06:45:15.301365 18954 ts_tablet_manager.cc:1905] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:15.301430 18256 catalog_manager.cc:4841] TS 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757): tablet cccf774ee5d9487cbe5c22e8de6b7196 (table test-table [id=7abdde8818ba42e3a1ebafd0c5019ef6]) successfully deleted
I20260810 06:45:15.301486 18750 tablet_service.cc:1467] Processing CreateTablet for tablet b673d2453b0e435d97d507937f335202 (DEFAULT_TABLE table=test-table [id=8b3a0db0e1e44ee1ba6e262401218529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:15.301594 18750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b673d2453b0e435d97d507937f335202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:15.301762 18427 tablet_service.cc:1467] Processing CreateTablet for tablet b673d2453b0e435d97d507937f335202 (DEFAULT_TABLE table=test-table [id=8b3a0db0e1e44ee1ba6e262401218529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:15.301884 18427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b673d2453b0e435d97d507937f335202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:15.302843 18954 ts_tablet_manager.cc:1918] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:15.302903 18954 log.cc:1196] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-0/wal/wals/cccf774ee5d9487cbe5c22e8de6b7196
I20260810 06:45:15.303117 18954 ts_tablet_manager.cc:1939] T cccf774ee5d9487cbe5c22e8de6b7196 P f598e218e9f3407983f409a6d100404b: Deleting consensus metadata
I20260810 06:45:15.303300 18965 tablet_bootstrap.cc:492] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3: Bootstrap starting.
I20260810 06:45:15.303687 18965 tablet_bootstrap.cc:654] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:15.304044 18969 tablet_bootstrap.cc:492] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b: Bootstrap starting.
I20260810 06:45:15.304394 18969 tablet_bootstrap.cc:654] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:15.304463 18965 tablet_bootstrap.cc:492] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3: No bootstrap required, opened a new log
I20260810 06:45:15.304517 18965 ts_tablet_manager.cc:1397] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:15.304636 18965 raft_consensus.cc:348] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:15.304711 18965 raft_consensus.cc:374] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:15.304740 18965 raft_consensus.cc:729] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36667d5b23cc42d2b23692f7553dcee3, State: Initialized, Role: FOLLOWER
I20260810 06:45:15.304783 18965 consensus_queue.cc:260] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:15.304889 18965 ts_tablet_manager.cc:1428] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:15.305279 18969 tablet_bootstrap.cc:492] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b: No bootstrap required, opened a new log
I20260810 06:45:15.305334 18969 ts_tablet_manager.cc:1397] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:15.305308 18590 tablet_service.cc:1467] Processing CreateTablet for tablet b673d2453b0e435d97d507937f335202 (DEFAULT_TABLE table=test-table [id=8b3a0db0e1e44ee1ba6e262401218529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:15.305433 18969 raft_consensus.cc:348] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:15.305455 18590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b673d2453b0e435d97d507937f335202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:15.305505 18969 raft_consensus.cc:374] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:15.305526 18969 raft_consensus.cc:729] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f598e218e9f3407983f409a6d100404b, State: Initialized, Role: FOLLOWER
I20260810 06:45:15.305569 18969 consensus_queue.cc:260] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:15.305657 18969 ts_tablet_manager.cc:1428] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:15.306562 18964 tablet_bootstrap.cc:492] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95: Bootstrap starting.
I20260810 06:45:15.306852 18964 tablet_bootstrap.cc:654] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:15.306857 18254 catalog_manager.cc:4841] TS f598e218e9f3407983f409a6d100404b (127.15.232.193:42985): tablet cccf774ee5d9487cbe5c22e8de6b7196 (table test-table [id=7abdde8818ba42e3a1ebafd0c5019ef6]) successfully deleted
I20260810 06:45:15.306931 18254 catalog_manager.cc:4841] TS 8d8e1a8ac5aa48aba9a19782fe026d95 (127.15.232.194:45517): tablet cccf774ee5d9487cbe5c22e8de6b7196 (table test-table [id=7abdde8818ba42e3a1ebafd0c5019ef6]) successfully deleted
I20260810 06:45:15.307547 18964 tablet_bootstrap.cc:492] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95: No bootstrap required, opened a new log
I20260810 06:45:15.307605 18964 ts_tablet_manager.cc:1397] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:15.307739 18964 raft_consensus.cc:348] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:15.307808 18964 raft_consensus.cc:374] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:15.307827 18964 raft_consensus.cc:729] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d8e1a8ac5aa48aba9a19782fe026d95, State: Initialized, Role: FOLLOWER
I20260810 06:45:15.307874 18964 consensus_queue.cc:260] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:15.307971 18964 ts_tablet_manager.cc:1428] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:15.451701 18936 raft_consensus.cc:482] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:15.451813 18936 raft_consensus.cc:504] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:15.452021 18936 leader_election.cc:288] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f598e218e9f3407983f409a6d100404b (127.15.232.193:42985), 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757)
I20260810 06:45:15.452334 18451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b673d2453b0e435d97d507937f335202" candidate_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f598e218e9f3407983f409a6d100404b" is_pre_election: true
I20260810 06:45:15.452353 18775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b673d2453b0e435d97d507937f335202" candidate_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36667d5b23cc42d2b23692f7553dcee3" is_pre_election: true
I20260810 06:45:15.452461 18775 raft_consensus.cc:2454] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d8e1a8ac5aa48aba9a19782fe026d95 in term 0.
I20260810 06:45:15.452469 18451 raft_consensus.cc:2454] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d8e1a8ac5aa48aba9a19782fe026d95 in term 0.
I20260810 06:45:15.452692 18536 leader_election.cc:302] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d8e1a8ac5aa48aba9a19782fe026d95, f598e218e9f3407983f409a6d100404b; no voters:
I20260810 06:45:15.452919 18936 raft_consensus.cc:2781] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:15.452986 18936 raft_consensus.cc:482] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:15.453020 18936 raft_consensus.cc:3037] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:15.453791 18936 raft_consensus.cc:504] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:15.453950 18936 leader_election.cc:288] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [CANDIDATE]: Term 1 election: Requested vote from peers f598e218e9f3407983f409a6d100404b (127.15.232.193:42985), 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757)
I20260810 06:45:15.454114 18775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b673d2453b0e435d97d507937f335202" candidate_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36667d5b23cc42d2b23692f7553dcee3"
I20260810 06:45:15.454149 18451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b673d2453b0e435d97d507937f335202" candidate_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f598e218e9f3407983f409a6d100404b"
I20260810 06:45:15.454224 18451 raft_consensus.cc:3037] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:15.454222 18775 raft_consensus.cc:3037] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:15.455092 18775 raft_consensus.cc:2454] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d8e1a8ac5aa48aba9a19782fe026d95 in term 1.
I20260810 06:45:15.455229 18451 raft_consensus.cc:2454] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d8e1a8ac5aa48aba9a19782fe026d95 in term 1.
I20260810 06:45:15.455366 18538 leader_election.cc:302] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36667d5b23cc42d2b23692f7553dcee3, 8d8e1a8ac5aa48aba9a19782fe026d95; no voters:
I20260810 06:45:15.455461 18936 raft_consensus.cc:2781] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:15.455529 18936 raft_consensus.cc:686] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 LEADER]: Becoming Leader. State: Replica: 8d8e1a8ac5aa48aba9a19782fe026d95, State: Running, Role: LEADER
I20260810 06:45:15.455585 18936 consensus_queue.cc:237] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } }
I20260810 06:45:15.456280 18271 catalog_manager.cc:5495] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d8e1a8ac5aa48aba9a19782fe026d95 (127.15.232.194). New cstate: current_term: 1 leader_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d8e1a8ac5aa48aba9a19782fe026d95" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 } health_report { overall_health: UNKNOWN } } }
I20260810 06:45:15.501209 18451 raft_consensus.cc:1261] T b673d2453b0e435d97d507937f335202 P f598e218e9f3407983f409a6d100404b [term 1 FOLLOWER]: Refusing update from remote peer 8d8e1a8ac5aa48aba9a19782fe026d95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:15.501221 18775 raft_consensus.cc:1261] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Refusing update from remote peer 8d8e1a8ac5aa48aba9a19782fe026d95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:15.501489 18936 consensus_queue.cc:1035] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36667d5b23cc42d2b23692f7553dcee3" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:15.501596 18930 consensus_queue.cc:1035] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f598e218e9f3407983f409a6d100404b" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:45:15.507472 18928 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 06:45:17.026968 18270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:17.027084 18270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:17.028039 18270 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0f6671e9af244c8adbc4d96c888c256: Sending DeleteTablet for 3 replicas of tablet b673d2453b0e435d97d507937f335202
I20260810 06:45:17.028357 18590 tablet_service.cc:1514] Processing DeleteTablet for tablet b673d2453b0e435d97d507937f335202 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:17 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:57334
I20260810 06:45:17.028357 18427 tablet_service.cc:1514] Processing DeleteTablet for tablet b673d2453b0e435d97d507937f335202 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:17 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:42382
I20260810 06:45:17.028694 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 18355
I20260810 06:45:17.029277 18750 tablet_service.cc:1514] Processing DeleteTablet for tablet b673d2453b0e435d97d507937f335202 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:17 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:59588
I20260810 06:45:17.029466 19011 tablet_replica.cc:330] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3: stopping tablet replica
I20260810 06:45:17.029525 19011 raft_consensus.cc:2229] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:17.029562 19011 raft_consensus.cc:2258] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:17.029866 19011 ts_tablet_manager.cc:1905] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:17.032270 19011 ts_tablet_manager.cc:1918] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:45:17.032357 19011 log.cc:1196] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/b673d2453b0e435d97d507937f335202
I20260810 06:45:17.032641 19011 ts_tablet_manager.cc:1939] T b673d2453b0e435d97d507937f335202 P 36667d5b23cc42d2b23692f7553dcee3: Deleting consensus metadata
I20260810 06:45:17.033142 18256 catalog_manager.cc:4841] TS 36667d5b23cc42d2b23692f7553dcee3 (127.15.232.195:33757): tablet b673d2453b0e435d97d507937f335202 (table test-table [id=8b3a0db0e1e44ee1ba6e262401218529]) successfully deleted
I20260810 06:45:17.034988 19009 tablet_replica.cc:330] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95: stopping tablet replica
I20260810 06:45:17.035079 19009 raft_consensus.cc:2229] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:17.035154 19009 raft_consensus.cc:2258] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:17.035461 19009 ts_tablet_manager.cc:1905] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:17.037030 19009 ts_tablet_manager.cc:1918] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:45:17.037117 19009 log.cc:1196] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/b673d2453b0e435d97d507937f335202
I20260810 06:45:17.037403 19009 ts_tablet_manager.cc:1939] T b673d2453b0e435d97d507937f335202 P 8d8e1a8ac5aa48aba9a19782fe026d95: Deleting consensus metadata
I20260810 06:45:17.037813 18254 catalog_manager.cc:4841] TS 8d8e1a8ac5aa48aba9a19782fe026d95 (127.15.232.194:45517): tablet b673d2453b0e435d97d507937f335202 (table test-table [id=8b3a0db0e1e44ee1ba6e262401218529]) successfully deleted
W20260810 06:45:17.040398 18254 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:42985 (error 108)
I20260810 06:45:17.040596 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 18516
I20260810 06:45:17.043147 19013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:17.042723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:42985 (local address 127.0.0.1:36400)
0810 06:45:17.043043 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:17.043100 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":264,"threads_started":1}
W20260810 06:45:17.043277 18254 catalog_manager.cc:4584] TS f598e218e9f3407983f409a6d100404b (127.15.232.193:42985): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b673d2453b0e435d97d507937f335202: Network error: Client connection negotiation failed: client connection to 127.15.232.193:42985: connect: Connection refused (error 111)
I20260810 06:45:17.052126 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 18678
I20260810 06:45:17.061225 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 18238
2026-08-10T06:45:17Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6304 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:17 dist-test-slave-vxj2 krb5kdc[19016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:17 dist-test-slave-vxj2 krb5kdc[19016](info): set up 2 sockets
Aug 10 06:45:17 dist-test-slave-vxj2 krb5kdc[19016](info): commencing operation
krb5kdc: starting...
W20260810 06:45:18.219182 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:18 dist-test-slave-vxj2 krb5kdc[19016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:45:18.344810 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:46373
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:46373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:18.430858 19032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:18.431075 19032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:18.431098 19032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:18.432196 19032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:18.432232 19032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:18.432246 19032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:18.432260 19032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:18.432271 19032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:18.433713 19032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:46373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:46373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:18.433923 19032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:18.434109 19032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:18.439196 19042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:18.439205 19040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:18.439415 19032 server_base.cc:1034] running on GCE node
W20260810 06:45:18.439247 19039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:18.439779 19032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:18.440042 19032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:18.441218 19032 hybrid_clock.cc:648] HybridClock initialized: now 1786344318441199 us; error 43 us; skew 500 ppm
Aug 10 06:45:18 dist-test-slave-vxj2 krb5kdc[19016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:18.443810 19032 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:45:18.444219 19032 webserver.cc:460] Webserver started at http://127.15.232.254:39603/ using document root <none> and password file <none>
I20260810 06:45:18.444464 19032 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:18.444512 19032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:18.444655 19032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:18.445814 19032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "52d5acf47aad4f66afa78e972e5d3eb8"
format_stamp: "Formatted at 2026-08-10 06:45:18 on dist-test-slave-vxj2"
server_key: "3d2bdfe690b98d9b80fd7a8eb679fa8f"
server_key_iv: "ecfa634e789895896eeb2bc2e26857ad"
server_key_version: "encryptionkey@0"
I20260810 06:45:18.446192 19032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "52d5acf47aad4f66afa78e972e5d3eb8"
format_stamp: "Formatted at 2026-08-10 06:45:18 on dist-test-slave-vxj2"
server_key: "3d2bdfe690b98d9b80fd7a8eb679fa8f"
server_key_iv: "ecfa634e789895896eeb2bc2e26857ad"
server_key_version: "encryptionkey@0"
I20260810 06:45:18.447646 19032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:18.448433 19049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:18.448684 19032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:18.448784 19032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "52d5acf47aad4f66afa78e972e5d3eb8"
format_stamp: "Formatted at 2026-08-10 06:45:18 on dist-test-slave-vxj2"
server_key: "3d2bdfe690b98d9b80fd7a8eb679fa8f"
server_key_iv: "ecfa634e789895896eeb2bc2e26857ad"
server_key_version: "encryptionkey@0"
I20260810 06:45:18.448860 19032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:18.462574 19032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:18.465058 19032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:18.465283 19032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:18.469323 19032 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:46373
I20260810 06:45:18.469420 19108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:46373 every 8 connection(s)
I20260810 06:45:18.469779 19032 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:18.470263 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 19032
I20260810 06:45:18.470378 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:18.470573 16291 external_mini_cluster.cc:1428] Setting key 1701f5ccba93a7b1aad750a49c53d0a5
I20260810 06:45:18.470865 19109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:18.473658 19109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8: Bootstrap starting.
I20260810 06:45:18.474321 19109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:18.474612 19109 log.cc:824] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:18.475303 19109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8: No bootstrap required, opened a new log
Aug 10 06:45:18 dist-test-slave-vxj2 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:18.477445 19109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d5acf47aad4f66afa78e972e5d3eb8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 46373 } }
I20260810 06:45:18.477629 19109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:18.477669 19109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d5acf47aad4f66afa78e972e5d3eb8, State: Initialized, Role: FOLLOWER
I20260810 06:45:18.477823 19109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d5acf47aad4f66afa78e972e5d3eb8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 46373 } }
I20260810 06:45:18.477921 19109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:18.477946 19109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:18.478021 19109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:18.478559 19109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d5acf47aad4f66afa78e972e5d3eb8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 46373 } }
I20260810 06:45:18.478646 19109 leader_election.cc:302] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52d5acf47aad4f66afa78e972e5d3eb8; no voters:
I20260810 06:45:18.478881 19109 leader_election.cc:288] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:18.479081 19117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:18.479193 19109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:18.479261 19117 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [term 1 LEADER]: Becoming Leader. State: Replica: 52d5acf47aad4f66afa78e972e5d3eb8, State: Running, Role: LEADER
I20260810 06:45:18.479391 19117 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d5acf47aad4f66afa78e972e5d3eb8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 46373 } }
I20260810 06:45:18.479768 19115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52d5acf47aad4f66afa78e972e5d3eb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d5acf47aad4f66afa78e972e5d3eb8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 46373 } } }
I20260810 06:45:18.479866 19115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:18.480065 19115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52d5acf47aad4f66afa78e972e5d3eb8. Latest consensus state: current_term: 1 leader_uuid: "52d5acf47aad4f66afa78e972e5d3eb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d5acf47aad4f66afa78e972e5d3eb8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 46373 } } }
I20260810 06:45:18.480162 19127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:18.480188 19115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:18.480814 19127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:18.482182 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:18.471219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36882 (local address 127.15.232.254:46373)
0810 06:45:18.471392 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:18.471401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:18.471436 (+ 35us) server_negotiation.cc:407] Connection header received
0810 06:45:18.471860 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:18.471867 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:18.471922 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:18.472072 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:18.472733 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.473771 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.474562 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.475017 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.475138 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:18.477883 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:18.477889 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:18.477889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:18.477905 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:18.480266 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:18.480716 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:18.480719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:18.480719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:18.480825 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:18.481142 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:18.481143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:18.481143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:18.481502 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:18.481641 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:18.481817 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 06:45:18.481959 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260810 06:45:18.482818 19127 catalog_manager.cc:1347] Generated new cluster ID: 123ca910d34f487b9ad699bc6af868ef
I20260810 06:45:18.482877 19127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:18.495604 19127 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:18.496199 19127 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:18.502940 19127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52d5acf47aad4f66afa78e972e5d3eb8: Generated new TSK 0
I20260810 06:45:18.503182 19127 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:45:18.536684 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:46373
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:18.622620 19141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:18.622881 19141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:18.622908 19141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:18.624076 19141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:18.624119 19141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:18.624145 19141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:18.625656 19141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:46373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:18.625897 19141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:18.626148 19141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:18.631678 19151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:18.631693 19148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:18.631765 19141 server_base.cc:1034] running on GCE node
W20260810 06:45:18.631678 19149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:18.632114 19141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:18.632316 19141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:18.633455 19141 hybrid_clock.cc:648] HybridClock initialized: now 1786344318633431 us; error 37 us; skew 500 ppm
Aug 10 06:45:18 dist-test-slave-vxj2 krb5kdc[19016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:18.636178 19141 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:45:18.636620 19141 webserver.cc:460] Webserver started at http://127.15.232.193:33109/ using document root <none> and password file <none>
I20260810 06:45:18.636857 19141 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:18.636904 19141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:18.637066 19141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:18.638195 19141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "ed00df7b93c7425b93f6c79d177420d9"
format_stamp: "Formatted at 2026-08-10 06:45:18 on dist-test-slave-vxj2"
server_key: "6eaee1d272d928c22edc24557656bdaa"
server_key_iv: "824617426ac903159c3b391db31a057d"
server_key_version: "encryptionkey@0"
I20260810 06:45:18.638593 19141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "ed00df7b93c7425b93f6c79d177420d9"
format_stamp: "Formatted at 2026-08-10 06:45:18 on dist-test-slave-vxj2"
server_key: "6eaee1d272d928c22edc24557656bdaa"
server_key_iv: "824617426ac903159c3b391db31a057d"
server_key_version: "encryptionkey@0"
I20260810 06:45:18.639992 19141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:18.640735 19160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:18.640923 19141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:18.641031 19141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "ed00df7b93c7425b93f6c79d177420d9"
format_stamp: "Formatted at 2026-08-10 06:45:18 on dist-test-slave-vxj2"
server_key: "6eaee1d272d928c22edc24557656bdaa"
server_key_iv: "824617426ac903159c3b391db31a057d"
server_key_version: "encryptionkey@0"
I20260810 06:45:18.641108 19141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:18.649134 19141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:18.651684 19141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:18.651858 19141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:18.652097 19141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:18.652444 19141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:18.652501 19141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:18.652545 19141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:18.652572 19141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:18.658689 19141 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:36549
I20260810 06:45:18.658741 19291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:36549 every 8 connection(s)
I20260810 06:45:18.659062 19141 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:18.661943 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 19141
I20260810 06:45:18.662019 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:18.662205 16291 external_mini_cluster.cc:1428] Setting key 4484cbf858f302e804f60e7f5c7c9780
Aug 10 06:45:18 dist-test-slave-vxj2 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:18.669835 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:18.659794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:39537 (local address 127.15.232.254:46373)
0810 06:45:18.659870 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:18.659872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:18.660081 (+ 209us) server_negotiation.cc:407] Connection header received
0810 06:45:18.660582 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:18.660583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:18.660594 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:18.660648 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:18.661666 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.662332 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.663721 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.663927 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.663985 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:18.666510 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:18.666512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:18.666512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:18.666522 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:18.668271 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:18.668712 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:18.668713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:18.668714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:18.668787 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:18.669052 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:18.669053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:18.669053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:18.669247 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:18.669329 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:18.669593 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 06:45:18.669697 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:45:18.669840 19294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:18.659772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:46373 (local address 127.15.232.193:39537)
0810 06:45:18.659970 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:18.660005 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:18.660492 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:18.660703 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:18.660706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:18.661354 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:18.661560 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:18.661565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.662432 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.662433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:18.663630 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:18.663634 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.664015 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.664017 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:18.664169 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:18.664681 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:18.664689 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:18.666390 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:18.668384 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:18.668392 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:18.668394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:18.668629 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:18.668879 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:18.668879 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:18.668879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:18.668980 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:18.669317 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:18.669322 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:18.669450 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 06:45:18.669538 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 06:45:18.669665 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":115,"threads_started":1}
I20260810 06:45:18.670506 19292 heartbeater.cc:347] Connected to a master server at 127.15.232.254:46373
I20260810 06:45:18.670608 19292 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:18.670815 19292 heartbeater.cc:511] Master 127.15.232.254:46373 requested a full tablet report, sending...
I20260810 06:45:18.671381 19070 ts_manager.cc:194] Registered new tserver with Master: ed00df7b93c7425b93f6c79d177420d9 (127.15.232.193:36549)
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
I20260810 06:45:18.672043 19070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:39537
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:45:18.693543 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:46373
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:18.776609 19297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:18.776870 19297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:18.776908 19297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:18.778097 19297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:18.778177 19297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:18.778220 19297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:18.779826 19297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:46373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:18.780109 19297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:18.780395 19297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:18.785620 19303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:18.785652 19302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:18.785643 19306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:18.785667 19297 server_base.cc:1034] running on GCE node
I20260810 06:45:18.786058 19297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:18.786280 19297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:18.787451 19297 hybrid_clock.cc:648] HybridClock initialized: now 1786344318787427 us; error 47 us; skew 500 ppm
Aug 10 06:45:18 dist-test-slave-vxj2 krb5kdc[19016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:18.790041 19297 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:45:18.790503 19297 webserver.cc:460] Webserver started at http://127.15.232.194:44449/ using document root <none> and password file <none>
I20260810 06:45:18.790690 19297 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:18.790731 19297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:18.790824 19297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:18.791850 19297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "24b2016cf2eb4763b6851ff44882aa6b"
format_stamp: "Formatted at 2026-08-10 06:45:18 on dist-test-slave-vxj2"
server_key: "93c24a044d62f8d0a3f49dcd94aa91f0"
server_key_iv: "beb8941897f651cfdd0adc6fdd805476"
server_key_version: "encryptionkey@0"
I20260810 06:45:18.792157 19297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "24b2016cf2eb4763b6851ff44882aa6b"
format_stamp: "Formatted at 2026-08-10 06:45:18 on dist-test-slave-vxj2"
server_key: "93c24a044d62f8d0a3f49dcd94aa91f0"
server_key_iv: "beb8941897f651cfdd0adc6fdd805476"
server_key_version: "encryptionkey@0"
I20260810 06:45:18.793463 19297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:18.794162 19313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:18.794349 19297 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:45:18.794440 19297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "24b2016cf2eb4763b6851ff44882aa6b"
format_stamp: "Formatted at 2026-08-10 06:45:18 on dist-test-slave-vxj2"
server_key: "93c24a044d62f8d0a3f49dcd94aa91f0"
server_key_iv: "beb8941897f651cfdd0adc6fdd805476"
server_key_version: "encryptionkey@0"
I20260810 06:45:18.794495 19297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:18.802475 19297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:18.804487 19297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:18.804600 19297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:18.804782 19297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:18.805105 19297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:18.805140 19297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:18.805217 19297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:18.805253 19297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:18.811098 19297 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:40985
I20260810 06:45:18.811185 19443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:40985 every 8 connection(s)
I20260810 06:45:18.811503 19297 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:45:18 dist-test-slave-vxj2 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:18.818732 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 19297
I20260810 06:45:18.818835 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:18.819067 16291 external_mini_cluster.cc:1428] Setting key b9e8602e6748d2fa89deb7e7be80bbda
I20260810 06:45:18.820999 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:18.812278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:41073 (local address 127.15.232.254:46373)
0810 06:45:18.812348 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:18.812350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:18.812577 (+ 227us) server_negotiation.cc:407] Connection header received
0810 06:45:18.813108 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:18.813110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:18.813122 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:18.813228 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:18.814206 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.814722 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.815738 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.815951 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.816039 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:18.818086 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:18.818088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:18.818089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:18.818096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:18.819532 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:18.819945 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:18.819947 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:18.819947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:18.820011 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:18.820279 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:18.820280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:18.820281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:18.820402 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:18.820479 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:18.820810 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 06:45:18.820896 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:45:18.821297 19448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:18.812242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:46373 (local address 127.15.232.194:41073)
0810 06:45:18.812443 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:18.812485 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:18.812998 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:18.813330 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:18.813335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:18.813911 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:18.814101 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:18.814106 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.814836 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.814837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:18.815630 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:18.815633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.816051 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.816052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:18.816183 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:18.816634 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:18.816643 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:18.817986 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:18.819630 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:18.819637 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:18.819640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:18.819866 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:18.820081 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:18.820082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:18.820082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:18.820193 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:18.820470 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:18.820477 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:18.820617 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 06:45:18.820728 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 06:45:18.820899 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":101,"threads_started":1}
I20260810 06:45:18.821765 19444 heartbeater.cc:347] Connected to a master server at 127.15.232.254:46373
I20260810 06:45:18.821857 19444 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:18.822049 19444 heartbeater.cc:511] Master 127.15.232.254:46373 requested a full tablet report, sending...
I20260810 06:45:18.822500 19070 ts_manager.cc:194] Registered new tserver with Master: 24b2016cf2eb4763b6851ff44882aa6b (127.15.232.194:40985)
I20260810 06:45:18.823017 19070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:41073
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:45:18.848802 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:46373
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:18.931754 19452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:18.931980 19452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:18.932004 19452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:18.933192 19452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:18.933236 19452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:18.933264 19452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:18.934769 19452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:46373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:18.934962 19452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:18.935140 19452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:18.940096 19459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:18.940227 19464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:18.940096 19460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:18.940382 19452 server_base.cc:1034] running on GCE node
I20260810 06:45:18.940589 19452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:18.940834 19452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:18.942008 19452 hybrid_clock.cc:648] HybridClock initialized: now 1786344318942000 us; error 39 us; skew 500 ppm
Aug 10 06:45:18 dist-test-slave-vxj2 krb5kdc[19016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:18.944734 19452 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:45:18.945308 19452 webserver.cc:460] Webserver started at http://127.15.232.195:39295/ using document root <none> and password file <none>
I20260810 06:45:18.945557 19452 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:18.945627 19452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:18.945770 19452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:18.946882 19452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "f0da91b28023421dae9269899264a94f"
format_stamp: "Formatted at 2026-08-10 06:45:18 on dist-test-slave-vxj2"
server_key: "6f7ff6a3bad612cba73282d1fb91ef2c"
server_key_iv: "9ab4c5a167d0a4ca281c8125698375b7"
server_key_version: "encryptionkey@0"
I20260810 06:45:18.947269 19452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "f0da91b28023421dae9269899264a94f"
format_stamp: "Formatted at 2026-08-10 06:45:18 on dist-test-slave-vxj2"
server_key: "6f7ff6a3bad612cba73282d1fb91ef2c"
server_key_iv: "9ab4c5a167d0a4ca281c8125698375b7"
server_key_version: "encryptionkey@0"
I20260810 06:45:18.948650 19452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:18.949468 19474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:18.949651 19452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:18.949754 19452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "f0da91b28023421dae9269899264a94f"
format_stamp: "Formatted at 2026-08-10 06:45:18 on dist-test-slave-vxj2"
server_key: "6f7ff6a3bad612cba73282d1fb91ef2c"
server_key_iv: "9ab4c5a167d0a4ca281c8125698375b7"
server_key_version: "encryptionkey@0"
I20260810 06:45:18.949838 19452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:18.963483 19452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:18.965692 19452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:18.965874 19452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:18.966140 19452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:18.966553 19452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:18.966616 19452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:18.966652 19452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:18.966679 19452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:18.973016 19452 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:33697
I20260810 06:45:18.973075 19614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:33697 every 8 connection(s)
I20260810 06:45:18.973450 19452 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:18.974331 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 19452
I20260810 06:45:18.974443 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:18.974644 16291 external_mini_cluster.cc:1428] Setting key 4555dc8990fc38e18d18a8fbd1bbc506
Aug 10 06:45:18 dist-test-slave-vxj2 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:18.984020 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:18.974274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:60995 (local address 127.15.232.254:46373)
0810 06:45:18.974350 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:18.974353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:18.974532 (+ 179us) server_negotiation.cc:407] Connection header received
0810 06:45:18.975248 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:18.975249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:18.975262 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:18.975321 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:18.976626 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.977124 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.978326 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.978533 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.978593 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:18.981099 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:18.981101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:18.981101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:18.981118 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:18.982583 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:18.983016 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:18.983017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:18.983018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:18.983080 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:18.983342 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:18.983343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:18.983343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:18.983466 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:18.983543 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:18.983829 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 06:45:18.983925 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:45:18.984222 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:18.974223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:46373 (local address 127.15.232.195:60995)
0810 06:45:18.974398 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:18.974446 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:18.975068 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:18.975461 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:18.975467 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:18.976317 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:18.976537 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:18.976542 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.977277 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.977278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:18.978229 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:18.978242 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:18.978618 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:18.978619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:18.978732 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:18.979291 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:18.979300 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:18.980990 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:18.982672 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:18.982680 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:18.982682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:18.982923 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:18.983152 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:18.983153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:18.983154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:18.983267 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:18.983534 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:18.983542 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:18.983689 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 06:45:18.983796 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 06:45:18.983966 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260810 06:45:18.984701 19615 heartbeater.cc:347] Connected to a master server at 127.15.232.254:46373
I20260810 06:45:18.984838 19615 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:18.985028 19615 heartbeater.cc:511] Master 127.15.232.254:46373 requested a full tablet report, sending...
I20260810 06:45:18.985522 19070 ts_manager.cc:194] Registered new tserver with Master: f0da91b28023421dae9269899264a94f (127.15.232.195:33697)
I20260810 06:45:18.985918 19070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:60995
I20260810 06:45:18.986984 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:45:18.998591 19113 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:18.997571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36888 (local address 127.15.232.254:46373)
0810 06:45:18.997640 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:18.997642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:18.997753 (+ 111us) server_negotiation.cc:407] Connection header received
0810 06:45:18.997835 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:18.997836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:18.997848 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:18.997915 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:18.998509 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36888: BlockingRecv error: recv got EOF from 127.0.0.1:36888 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:45:18.999539 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 19141
I20260810 06:45:19.014444 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 19297
I20260810 06:45:19.022634 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 19452
I20260810 06:45:19.029964 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 19032
2026-08-10T06:45:19Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1966 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:19 dist-test-slave-vxj2 krb5kdc[19627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:19 dist-test-slave-vxj2 krb5kdc[19627](info): set up 2 sockets
Aug 10 06:45:19 dist-test-slave-vxj2 krb5kdc[19627](info): commencing operation
krb5kdc: starting...
W20260810 06:45:20.237608 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:20 dist-test-slave-vxj2 krb5kdc[19627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:45:20.370195 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:33707
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:33707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:20.463155 19641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:20.463397 19641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:20.463424 19641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:20.464733 19641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:20.464782 19641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:20.464799 19641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:20.464813 19641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:20.464828 19641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:20.466573 19641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:33707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:33707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:20.466861 19641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:20.467123 19641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:20.472539 19646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:20.472586 19648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:20.472541 19651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:20.472679 19641 server_base.cc:1034] running on GCE node
I20260810 06:45:20.473191 19641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:20.473508 19641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:20.474680 19641 hybrid_clock.cc:648] HybridClock initialized: now 1786344320474666 us; error 30 us; skew 500 ppm
Aug 10 06:45:20 dist-test-slave-vxj2 krb5kdc[19627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344320, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:20.482739 19641 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:45:20.483345 19641 webserver.cc:460] Webserver started at http://127.15.232.254:38857/ using document root <none> and password file <none>
I20260810 06:45:20.483618 19641 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:20.483691 19641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:20.483847 19641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:20.484946 19641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "5229034cd6034171a061b74fbce6cc47"
format_stamp: "Formatted at 2026-08-10 06:45:20 on dist-test-slave-vxj2"
server_key: "a93727af7d8ae051ebc7c605296681c0"
server_key_iv: "f40a280e5d992e290079f9ee38e403b2"
server_key_version: "encryptionkey@0"
I20260810 06:45:20.485345 19641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "5229034cd6034171a061b74fbce6cc47"
format_stamp: "Formatted at 2026-08-10 06:45:20 on dist-test-slave-vxj2"
server_key: "a93727af7d8ae051ebc7c605296681c0"
server_key_iv: "f40a280e5d992e290079f9ee38e403b2"
server_key_version: "encryptionkey@0"
I20260810 06:45:20.486749 19641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:20.487578 19658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:20.487789 19641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:20.487921 19641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "5229034cd6034171a061b74fbce6cc47"
format_stamp: "Formatted at 2026-08-10 06:45:20 on dist-test-slave-vxj2"
server_key: "a93727af7d8ae051ebc7c605296681c0"
server_key_iv: "f40a280e5d992e290079f9ee38e403b2"
server_key_version: "encryptionkey@0"
I20260810 06:45:20.488000 19641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:20.504490 19641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:20.506889 19641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:20.507073 19641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:20.511473 19641 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:33707
I20260810 06:45:20.511510 19718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:33707 every 8 connection(s)
I20260810 06:45:20.512102 19641 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:20.512876 19719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:20.515295 19719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47: Bootstrap starting.
I20260810 06:45:20.515866 19719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:20.516207 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 19641
I20260810 06:45:20.516208 19719 log.cc:824] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:20.516310 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:20.516515 16291 external_mini_cluster.cc:1428] Setting key 831d0d8557a0ca7bc1edec2f034cabea
I20260810 06:45:20.517136 19719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47: No bootstrap required, opened a new log
I20260810 06:45:20.519980 19719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5229034cd6034171a061b74fbce6cc47" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 33707 } }
I20260810 06:45:20.520156 19719 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:20.520195 19719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5229034cd6034171a061b74fbce6cc47, State: Initialized, Role: FOLLOWER
I20260810 06:45:20.520321 19719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5229034cd6034171a061b74fbce6cc47" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 33707 } }
I20260810 06:45:20.520416 19719 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:20.520452 19719 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:20.520504 19719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:20.521266 19719 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5229034cd6034171a061b74fbce6cc47" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 33707 } }
I20260810 06:45:20.521391 19719 leader_election.cc:302] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5229034cd6034171a061b74fbce6cc47; no voters:
I20260810 06:45:20.521585 19719 leader_election.cc:288] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:20.521660 19725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:20.521864 19719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:20.521941 19725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [term 1 LEADER]: Becoming Leader. State: Replica: 5229034cd6034171a061b74fbce6cc47, State: Running, Role: LEADER
I20260810 06:45:20.522087 19725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5229034cd6034171a061b74fbce6cc47" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 33707 } }
I20260810 06:45:20.522481 19725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5229034cd6034171a061b74fbce6cc47. Latest consensus state: current_term: 1 leader_uuid: "5229034cd6034171a061b74fbce6cc47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5229034cd6034171a061b74fbce6cc47" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 33707 } } }
I20260810 06:45:20.522634 19725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:20.522989 19734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:20.523504 19726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5229034cd6034171a061b74fbce6cc47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5229034cd6034171a061b74fbce6cc47" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 33707 } } }
I20260810 06:45:20.523787 19726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47 [sys.catalog]: This master's current role is: LEADER
Aug 10 06:45:20 dist-test-slave-vxj2 krb5kdc[19627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:20.523594 19734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:20.526827 19734 catalog_manager.cc:1347] Generated new cluster ID: 3f8643051df647f4a5a8de3507bd445b
I20260810 06:45:20.526897 19734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:20.529987 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:20.517071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49380 (local address 127.15.232.254:33707)
0810 06:45:20.517285 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:20.517295 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:20.517335 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:45:20.517856 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:20.517865 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:20.517930 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:20.518094 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:20.518726 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:20.520231 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:20.521450 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:20.522231 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:20.522387 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:20.525482 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:20.525486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:20.525487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:20.525500 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:20.527893 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:20.528351 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:20.528354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:20.528355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:20.528496 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:20.528775 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:20.528776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:20.528777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:20.529145 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:20.529370 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:20.529463 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 06:45:20.529740 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":108,"threads_started":1}
I20260810 06:45:20.531051 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:45:20.549324 19734 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:20.550165 19734 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:20.554930 19734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5229034cd6034171a061b74fbce6cc47: Generated new TSK 0
I20260810 06:45:20.555147 19734 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:45:20.557750 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:20.553689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49396 (local address 127.15.232.254:33707)
0810 06:45:20.553743 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:20.553745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:20.553971 (+ 226us) server_negotiation.cc:407] Connection header received
0810 06:45:20.554048 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:20.554049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:20.554061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:20.554107 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:20.554592 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:20.555375 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:20.556154 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:20.556391 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:20.556478 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:20.557283 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:20.557284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:20.557284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:20.557481 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:20.557497 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:20.557504 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:20.557569 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:20.557634 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 06:45:20.557662 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:45:20.560940 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 19641
2026-08-10T06:45:20Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1529 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:20 dist-test-slave-vxj2 krb5kdc[19757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:20 dist-test-slave-vxj2 krb5kdc[19757](info): set up 2 sockets
Aug 10 06:45:20 dist-test-slave-vxj2 krb5kdc[19757](info): commencing operation
krb5kdc: starting...
W20260810 06:45:21.704294 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:21 dist-test-slave-vxj2 krb5kdc[19757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:45:21.832952 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:43497
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:43497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:21.916680 19771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:21.916913 19771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:21.916936 19771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:21.918160 19771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:21.918208 19771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:21.918222 19771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:21.918234 19771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:21.918292 19771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:21.919821 19771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:43497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:43497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:21.920097 19771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:21.920337 19771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:21.925427 19777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:21.925422 19776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:21.925450 19779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:21.925681 19771 server_base.cc:1034] running on GCE node
I20260810 06:45:21.925968 19771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:21.926294 19771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:21.927464 19771 hybrid_clock.cc:648] HybridClock initialized: now 1786344321927472 us; error 50 us; skew 500 ppm
Aug 10 06:45:21 dist-test-slave-vxj2 krb5kdc[19757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:21.930132 19771 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:45:21.930650 19771 webserver.cc:460] Webserver started at http://127.15.232.254:45629/ using document root <none> and password file <none>
I20260810 06:45:21.930913 19771 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:21.930985 19771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:21.931143 19771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:21.932226 19771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "43601632461c406a9b81bbf3ff10610d"
format_stamp: "Formatted at 2026-08-10 06:45:21 on dist-test-slave-vxj2"
server_key: "a995e8cac1ef0fea5d74a787f35b265a"
server_key_iv: "443160b3c5e93ba38309099e3634b0d6"
server_key_version: "encryptionkey@0"
I20260810 06:45:21.932618 19771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "43601632461c406a9b81bbf3ff10610d"
format_stamp: "Formatted at 2026-08-10 06:45:21 on dist-test-slave-vxj2"
server_key: "a995e8cac1ef0fea5d74a787f35b265a"
server_key_iv: "443160b3c5e93ba38309099e3634b0d6"
server_key_version: "encryptionkey@0"
I20260810 06:45:21.934141 19771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260810 06:45:21.935021 19787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:21.935261 19771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:21.935359 19771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "43601632461c406a9b81bbf3ff10610d"
format_stamp: "Formatted at 2026-08-10 06:45:21 on dist-test-slave-vxj2"
server_key: "a995e8cac1ef0fea5d74a787f35b265a"
server_key_iv: "443160b3c5e93ba38309099e3634b0d6"
server_key_version: "encryptionkey@0"
I20260810 06:45:21.935464 19771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:21.948432 19771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:21.950534 19771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:21.950709 19771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:21.955117 19771 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:43497
I20260810 06:45:21.955163 19848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:43497 every 8 connection(s)
I20260810 06:45:21.955556 19771 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:21.956729 19849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:21.958345 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 19771
I20260810 06:45:21.958461 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:21.958670 16291 external_mini_cluster.cc:1428] Setting key 83bfc2e0ebc525c0775e8dadd9710c70
I20260810 06:45:21.959581 19849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d: Bootstrap starting.
I20260810 06:45:21.960232 19849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:21.960593 19849 log.cc:824] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:21.961457 19849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d: No bootstrap required, opened a new log
I20260810 06:45:21.963599 19849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43601632461c406a9b81bbf3ff10610d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43497 } }
I20260810 06:45:21.963759 19849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:21.963785 19849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43601632461c406a9b81bbf3ff10610d, State: Initialized, Role: FOLLOWER
I20260810 06:45:21.963976 19849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43601632461c406a9b81bbf3ff10610d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43497 } }
I20260810 06:45:21.964102 19849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:21.964157 19849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:21.964213 19849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:21.964838 19849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43601632461c406a9b81bbf3ff10610d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43497 } }
I20260810 06:45:21.964973 19849 leader_election.cc:302] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43601632461c406a9b81bbf3ff10610d; no voters:
I20260810 06:45:21.965202 19849 leader_election.cc:288] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:21.965206 19854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:21.965372 19854 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [term 1 LEADER]: Becoming Leader. State: Replica: 43601632461c406a9b81bbf3ff10610d, State: Running, Role: LEADER
I20260810 06:45:21.965508 19849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:21.965530 19854 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43601632461c406a9b81bbf3ff10610d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43497 } }
Aug 10 06:45:21 dist-test-slave-vxj2 krb5kdc[19757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:21.966697 19856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43601632461c406a9b81bbf3ff10610d. Latest consensus state: current_term: 1 leader_uuid: "43601632461c406a9b81bbf3ff10610d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43601632461c406a9b81bbf3ff10610d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43497 } } }
I20260810 06:45:21.967028 19855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43601632461c406a9b81bbf3ff10610d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43601632461c406a9b81bbf3ff10610d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43497 } } }
I20260810 06:45:21.967126 19856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:21.967149 19855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:21.967335 19871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:21.967900 19871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:21.971443 19852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:21.959247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46916 (local address 127.15.232.254:43497)
0810 06:45:21.959419 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:21.959434 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:21.959460 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:45:21.959936 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:21.959943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:21.960000 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:21.960133 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:21.961011 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:21.962111 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:21.962898 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:21.963314 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:21.963447 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:21.966408 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:21.966414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:21.966415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:21.966429 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:21.968726 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:21.969985 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:21.969988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:21.969989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:21.970115 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:21.970370 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:21.970371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:21.970371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:21.970711 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:21.970870 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:21.971064 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 06:45:21.971221 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260810 06:45:21.971915 19871 catalog_manager.cc:1347] Generated new cluster ID: 65cf6deacd704ba09a228a234515f326
I20260810 06:45:21.971976 19871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:21.984767 19871 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:21.985410 19871 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:21.993042 19871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43601632461c406a9b81bbf3ff10610d: Generated new TSK 0
I20260810 06:45:21.993368 19871 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:45:22.024094 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:43497
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:22.108213 19877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:22.108471 19877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:22.108511 19877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:22.109709 19877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:22.109779 19877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:22.109825 19877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:22.111296 19877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:43497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:22.111560 19877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:22.111800 19877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:22.117151 19884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:22.117126 19885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:22.117121 19887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:22.117175 19877 server_base.cc:1034] running on GCE node
I20260810 06:45:22.117566 19877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:22.117856 19877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:22.119042 19877 hybrid_clock.cc:648] HybridClock initialized: now 1786344322119025 us; error 31 us; skew 500 ppm
Aug 10 06:45:22 dist-test-slave-vxj2 krb5kdc[19757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344322, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:22.122193 19877 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:45:22.122638 19877 webserver.cc:460] Webserver started at http://127.15.232.193:33689/ using document root <none> and password file <none>
I20260810 06:45:22.122864 19877 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:22.122910 19877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:22.123008 19877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:22.124152 19877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "38f6bc4aa35f470fa872f0f4a5ad912b"
format_stamp: "Formatted at 2026-08-10 06:45:22 on dist-test-slave-vxj2"
server_key: "33eb065f0fae6578d9b2626738d9f609"
server_key_iv: "6a3d7fcff9fa36a7b707444f49f5e93a"
server_key_version: "encryptionkey@0"
I20260810 06:45:22.124500 19877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "38f6bc4aa35f470fa872f0f4a5ad912b"
format_stamp: "Formatted at 2026-08-10 06:45:22 on dist-test-slave-vxj2"
server_key: "33eb065f0fae6578d9b2626738d9f609"
server_key_iv: "6a3d7fcff9fa36a7b707444f49f5e93a"
server_key_version: "encryptionkey@0"
I20260810 06:45:22.126029 19877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:22.126965 19894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:22.127215 19877 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:22.127314 19877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "38f6bc4aa35f470fa872f0f4a5ad912b"
format_stamp: "Formatted at 2026-08-10 06:45:22 on dist-test-slave-vxj2"
server_key: "33eb065f0fae6578d9b2626738d9f609"
server_key_iv: "6a3d7fcff9fa36a7b707444f49f5e93a"
server_key_version: "encryptionkey@0"
I20260810 06:45:22.127375 19877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:22.148051 19877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:22.150828 19877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:22.150964 19877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:22.151193 19877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:22.151571 19877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:22.151638 19877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:22.151676 19877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:22.151712 19877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:22.158280 19877 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:34431
I20260810 06:45:22.158350 20023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:34431 every 8 connection(s)
I20260810 06:45:22.158730 19877 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:22.160394 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 19877
I20260810 06:45:22.160476 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:22.160674 16291 external_mini_cluster.cc:1428] Setting key 19c12c7525844f52f398484d12f3dc23
Aug 10 06:45:22 dist-test-slave-vxj2 krb5kdc[19757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344322, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
I20260810 06:45:22.171217 19852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:22.159585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:42575 (local address 127.15.232.254:43497)
0810 06:45:22.159662 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:22.159664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:22.159959 (+ 295us) server_negotiation.cc:407] Connection header received
0810 06:45:22.160576 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:22.160577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:22.160591 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:22.160656 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:22.162142 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.162969 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.164658 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.164891 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.164955 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:22.167780 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:22.167782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:22.167782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:22.167790 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:22.169423 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:22.169981 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:22.169982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:22.169983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:22.170044 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:22.170383 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:22.170385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:22.170385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:22.170556 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:22.170647 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:22.171046 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 06:45:22.171139 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:45:22.171674 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:22.159567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:43497 (local address 127.15.232.193:42575)
0810 06:45:22.159830 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:22.159863 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:22.160468 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:22.160775 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:22.160780 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:22.161703 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:22.161982 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:22.161989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.163104 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.163105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:22.164555 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:22.164558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.164987 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.164988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:22.165140 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:22.165831 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:22.165843 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:22.167665 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:22.169542 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:22.169551 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:22.169553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:22.169896 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:22.170122 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:22.170123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:22.170124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:22.170291 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:22.170662 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:22.170670 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:22.170888 (+ 218us) client_negotiation.cc:769] Sending connection context
0810 06:45:22.171009 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 06:45:22.171165 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":94,"threads_started":1}
I20260810 06:45:22.172392 20024 heartbeater.cc:347] Connected to a master server at 127.15.232.254:43497
I20260810 06:45:22.172509 20024 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:22.172730 20024 heartbeater.cc:511] Master 127.15.232.254:43497 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
I20260810 06:45:22.173462 19806 ts_manager.cc:194] Registered new tserver with Master: 38f6bc4aa35f470fa872f0f4a5ad912b (127.15.232.193:34431)
I20260810 06:45:22.174104 19806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:42575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:45:22.193589 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:43497
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:22.282631 20029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:22.282883 20029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:22.282912 20029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:22.284184 20029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:22.284245 20029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:22.284276 20029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:22.285910 20029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:43497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:22.286187 20029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:22.286474 20029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:22.291872 20037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:22.291917 20034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:22.291937 20035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:22.292212 20029 server_base.cc:1034] running on GCE node
I20260810 06:45:22.292455 20029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:22.292713 20029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:22.293915 20029 hybrid_clock.cc:648] HybridClock initialized: now 1786344322293901 us; error 33 us; skew 500 ppm
Aug 10 06:45:22 dist-test-slave-vxj2 krb5kdc[19757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344322, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:22.296657 20029 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:45:22.297116 20029 webserver.cc:460] Webserver started at http://127.15.232.194:36675/ using document root <none> and password file <none>
I20260810 06:45:22.297392 20029 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:22.297443 20029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:22.297544 20029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:22.298754 20029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "36b6eec2d63b4c1b86c5731ed56126f4"
format_stamp: "Formatted at 2026-08-10 06:45:22 on dist-test-slave-vxj2"
server_key: "cf6297157d5f3d37fd6b29df5aee337c"
server_key_iv: "c2b2edf654ed5112a9592fe6deb667b6"
server_key_version: "encryptionkey@0"
I20260810 06:45:22.299166 20029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "36b6eec2d63b4c1b86c5731ed56126f4"
format_stamp: "Formatted at 2026-08-10 06:45:22 on dist-test-slave-vxj2"
server_key: "cf6297157d5f3d37fd6b29df5aee337c"
server_key_iv: "c2b2edf654ed5112a9592fe6deb667b6"
server_key_version: "encryptionkey@0"
I20260810 06:45:22.300614 20029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:22.301453 20044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:22.301631 20029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:22.301717 20029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "36b6eec2d63b4c1b86c5731ed56126f4"
format_stamp: "Formatted at 2026-08-10 06:45:22 on dist-test-slave-vxj2"
server_key: "cf6297157d5f3d37fd6b29df5aee337c"
server_key_iv: "c2b2edf654ed5112a9592fe6deb667b6"
server_key_version: "encryptionkey@0"
I20260810 06:45:22.301777 20029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:22.313982 20029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:22.315914 20029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:22.316026 20029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:22.316226 20029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:22.316519 20029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:22.316551 20029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:22.316574 20029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:22.316587 20029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:22.323045 20029 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:37219
I20260810 06:45:22.323155 20174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:37219 every 8 connection(s)
I20260810 06:45:22.323464 20029 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:22.329526 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 20029
I20260810 06:45:22.329628 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:22.329839 16291 external_mini_cluster.cc:1428] Setting key e548bd3f5775171dd74103f570c41956
Aug 10 06:45:22 dist-test-slave-vxj2 krb5kdc[19757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344322, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:22.334451 19852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:22.324395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:38027 (local address 127.15.232.254:43497)
0810 06:45:22.324446 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:22.324448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:22.324667 (+ 219us) server_negotiation.cc:407] Connection header received
0810 06:45:22.325291 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:22.325292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:22.325305 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:22.325376 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:22.326503 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.327042 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.328120 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.328337 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.328397 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:22.331015 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:22.331016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:22.331017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:22.331025 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:22.332870 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:22.333424 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:22.333426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:22.333426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:22.333486 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:22.333798 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:22.333799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:22.333799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:22.333913 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:22.333965 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:22.334308 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 06:45:22.334383 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:45:22.334751 20178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:22.324318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:43497 (local address 127.15.232.194:38027)
0810 06:45:22.324515 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:22.324561 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:22.325099 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:22.325455 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:22.325459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:22.326161 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:22.326370 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:22.326376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.327156 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.327157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:22.328031 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:22.328033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.328450 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.328451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:22.328560 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:22.329218 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:22.329230 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:22.330915 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:22.332958 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:22.332965 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:22.332969 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:22.333327 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:22.333561 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:22.333562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:22.333563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:22.333724 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:22.333981 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:22.333988 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:22.334164 (+ 176us) client_negotiation.cc:769] Sending connection context
0810 06:45:22.334274 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 06:45:22.334463 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260810 06:45:22.335364 20175 heartbeater.cc:347] Connected to a master server at 127.15.232.254:43497
I20260810 06:45:22.335480 20175 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:22.335687 20175 heartbeater.cc:511] Master 127.15.232.254:43497 requested a full tablet report, sending...
I20260810 06:45:22.336081 19806 ts_manager.cc:194] Registered new tserver with Master: 36b6eec2d63b4c1b86c5731ed56126f4 (127.15.232.194:37219)
I20260810 06:45:22.336479 19806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:38027
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:45:22.360002 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:43497
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:22.446869 20181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:22.447086 20181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:22.447110 20181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:22.448227 20181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:22.448266 20181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:22.448292 20181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:22.449709 20181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:43497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:22.449901 20181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:22.450076 20181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:22.455040 20189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:22.455124 20188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:22.455225 20191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:22.455462 20181 server_base.cc:1034] running on GCE node
I20260810 06:45:22.455780 20181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:22.456079 20181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:22.457237 20181 hybrid_clock.cc:648] HybridClock initialized: now 1786344322457189 us; error 65 us; skew 500 ppm
Aug 10 06:45:22 dist-test-slave-vxj2 krb5kdc[19757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344322, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:22.459986 20181 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:45:22.460450 20181 webserver.cc:460] Webserver started at http://127.15.232.195:37765/ using document root <none> and password file <none>
I20260810 06:45:22.460644 20181 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:22.460683 20181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:22.460772 20181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:22.461841 20181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "b9340cdae4ba489a90d80d27e528ff88"
format_stamp: "Formatted at 2026-08-10 06:45:22 on dist-test-slave-vxj2"
server_key: "90c3317722422cf870a752b9dc559341"
server_key_iv: "ed1990ba8125aafd2836213c8b690c19"
server_key_version: "encryptionkey@0"
I20260810 06:45:22.462147 20181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "b9340cdae4ba489a90d80d27e528ff88"
format_stamp: "Formatted at 2026-08-10 06:45:22 on dist-test-slave-vxj2"
server_key: "90c3317722422cf870a752b9dc559341"
server_key_iv: "ed1990ba8125aafd2836213c8b690c19"
server_key_version: "encryptionkey@0"
I20260810 06:45:22.463541 20181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:22.464232 20198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:22.464423 20181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:22.464509 20181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "b9340cdae4ba489a90d80d27e528ff88"
format_stamp: "Formatted at 2026-08-10 06:45:22 on dist-test-slave-vxj2"
server_key: "90c3317722422cf870a752b9dc559341"
server_key_iv: "ed1990ba8125aafd2836213c8b690c19"
server_key_version: "encryptionkey@0"
I20260810 06:45:22.464561 20181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:22.472721 20181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:22.474992 20181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:22.475107 20181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:22.475286 20181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:22.475615 20181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:22.475651 20181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:22.475672 20181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:22.475723 20181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:22.481810 20181 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:39629
I20260810 06:45:22.481918 20324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:39629 every 8 connection(s)
I20260810 06:45:22.482242 20181 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:22.485857 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 20181
I20260810 06:45:22.485960 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:22.486141 16291 external_mini_cluster.cc:1428] Setting key bae91b5d086806d25a8d7893f67fb96b
Aug 10 06:45:22 dist-test-slave-vxj2 krb5kdc[19757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344322, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:22.492427 19852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:22.483136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:48873 (local address 127.15.232.254:43497)
0810 06:45:22.483235 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:22.483237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:22.483386 (+ 149us) server_negotiation.cc:407] Connection header received
0810 06:45:22.483895 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:22.483896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:22.483906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:22.483960 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:22.485131 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.485698 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.487172 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.487378 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.487430 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:22.489805 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:22.489806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:22.489807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:22.489814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:22.491263 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:22.491631 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:22.491632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:22.491633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:22.491682 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:22.491896 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:22.491897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:22.491897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:22.491995 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:22.492045 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:22.492287 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 06:45:22.492344 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:45:22.492671 20327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:22.483035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:43497 (local address 127.15.232.195:48873)
0810 06:45:22.483236 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:22.483292 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:22.483773 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:22.484072 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:22.484076 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:22.484840 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:22.485047 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:22.485053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.485856 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.485857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:22.487068 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:22.487071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.487456 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.487457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:22.487570 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:22.488154 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:22.488166 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:22.489714 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:22.491341 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:22.491347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:22.491349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:22.491566 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:22.491743 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:22.491743 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:22.491743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:22.491841 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:22.492052 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:22.492058 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:22.492180 (+ 122us) client_negotiation.cc:769] Sending connection context
0810 06:45:22.492278 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 06:45:22.492401 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":68,"threads_started":1}
I20260810 06:45:22.493367 20325 heartbeater.cc:347] Connected to a master server at 127.15.232.254:43497
I20260810 06:45:22.493487 20325 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:22.493710 20325 heartbeater.cc:511] Master 127.15.232.254:43497 requested a full tablet report, sending...
I20260810 06:45:22.494151 19806 ts_manager.cc:194] Registered new tserver with Master: b9340cdae4ba489a90d80d27e528ff88 (127.15.232.195:39629)
I20260810 06:45:22.494551 19806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:48873
I20260810 06:45:22.498018 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:45:22 dist-test-slave-vxj2 krb5kdc[19757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344322, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 06:45:22 dist-test-slave-vxj2 krb5kdc[19757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344322, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:22.523063 19852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:22.515328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46920 (local address 127.15.232.254:43497)
0810 06:45:22.515427 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:22.515431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:22.515537 (+ 106us) server_negotiation.cc:407] Connection header received
0810 06:45:22.515642 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:22.515644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:22.515655 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:22.515704 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:22.516349 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.516875 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.517734 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:22.517934 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:22.518009 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:22.520384 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:22.520386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:22.520386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:22.520395 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:22.521756 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:22.522182 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:22.522183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:22.522184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:22.522244 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:22.522454 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:22.522455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:22.522455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:22.522617 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:22.522729 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:22.522864 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 06:45:22.522950 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 06:45:22.523336 19806 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46920
I20260810 06:45:22.523813 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 19877
I20260810 06:45:22.530746 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 20029
I20260810 06:45:22.540537 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 20181
I20260810 06:45:22.548763 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 19771
2026-08-10T06:45:22Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1988 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:22 dist-test-slave-vxj2 krb5kdc[20339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:22 dist-test-slave-vxj2 krb5kdc[20339](info): set up 2 sockets
Aug 10 06:45:22 dist-test-slave-vxj2 krb5kdc[20339](info): commencing operation
krb5kdc: starting...
W20260810 06:45:23.678634 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:23 dist-test-slave-vxj2 krb5kdc[20339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:45:23.808104 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:36733
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:36733
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:23.898345 20353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:23.898624 20353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:23.898665 20353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:23.899967 20353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:23.900050 20353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:23.900079 20353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:23.900115 20353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:23.900449 20353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:23.902215 20353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:36733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:36733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:23.902504 20353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:23.902743 20353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:23.907706 20360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:23.907706 20363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:23.907766 20361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:23.908144 20353 server_base.cc:1034] running on GCE node
I20260810 06:45:23.908368 20353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:23.908628 20353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:23.909775 20353 hybrid_clock.cc:648] HybridClock initialized: now 1786344323909777 us; error 69 us; skew 500 ppm
Aug 10 06:45:23 dist-test-slave-vxj2 krb5kdc[20339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344323, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:23.912322 20353 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:45:23.912811 20353 webserver.cc:460] Webserver started at http://127.15.232.254:38555/ using document root <none> and password file <none>
I20260810 06:45:23.913087 20353 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:23.913141 20353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:23.913341 20353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:23.914449 20353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "581403433207421290ba75c2fc084982"
format_stamp: "Formatted at 2026-08-10 06:45:23 on dist-test-slave-vxj2"
server_key: "b250b2d2493e7f269df722febcc550cf"
server_key_iv: "bc7045328e08a3ad1f8252207a6b7edd"
server_key_version: "encryptionkey@0"
I20260810 06:45:23.914824 20353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "581403433207421290ba75c2fc084982"
format_stamp: "Formatted at 2026-08-10 06:45:23 on dist-test-slave-vxj2"
server_key: "b250b2d2493e7f269df722febcc550cf"
server_key_iv: "bc7045328e08a3ad1f8252207a6b7edd"
server_key_version: "encryptionkey@0"
I20260810 06:45:23.916256 20353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:23.917088 20373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:23.917351 20353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:23.917440 20353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "581403433207421290ba75c2fc084982"
format_stamp: "Formatted at 2026-08-10 06:45:23 on dist-test-slave-vxj2"
server_key: "b250b2d2493e7f269df722febcc550cf"
server_key_iv: "bc7045328e08a3ad1f8252207a6b7edd"
server_key_version: "encryptionkey@0"
I20260810 06:45:23.917500 20353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:23.940475 20353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:23.943136 20353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:23.943274 20353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:23.947496 20353 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:36733
I20260810 06:45:23.947570 20432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:36733 every 8 connection(s)
I20260810 06:45:23.948124 20353 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:23.948812 20433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:23.951179 20433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982: Bootstrap starting.
I20260810 06:45:23.951838 20433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:23.952145 20433 log.cc:824] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:23.952960 20433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982: No bootstrap required, opened a new log
I20260810 06:45:23.953747 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 20353
I20260810 06:45:23.953861 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:23.954063 16291 external_mini_cluster.cc:1428] Setting key 987a98f86314550cb7dd08d496ef7ae5
I20260810 06:45:23.955822 20433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "581403433207421290ba75c2fc084982" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36733 } }
I20260810 06:45:23.955977 20433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:23.956002 20433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 581403433207421290ba75c2fc084982, State: Initialized, Role: FOLLOWER
I20260810 06:45:23.956173 20433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "581403433207421290ba75c2fc084982" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36733 } }
I20260810 06:45:23.956285 20433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:23.956336 20433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:23.956389 20433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:23.956970 20433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "581403433207421290ba75c2fc084982" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36733 } }
I20260810 06:45:23.957100 20433 leader_election.cc:302] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 581403433207421290ba75c2fc084982; no voters:
I20260810 06:45:23.957340 20433 leader_election.cc:288] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:23.957384 20438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:23.957506 20438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [term 1 LEADER]: Becoming Leader. State: Replica: 581403433207421290ba75c2fc084982, State: Running, Role: LEADER
I20260810 06:45:23.957597 20438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "581403433207421290ba75c2fc084982" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36733 } }
I20260810 06:45:23.957630 20433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:23.957942 20438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 581403433207421290ba75c2fc084982. Latest consensus state: current_term: 1 leader_uuid: "581403433207421290ba75c2fc084982" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "581403433207421290ba75c2fc084982" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36733 } } }
I20260810 06:45:23.958032 20438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:23.957989 20440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "581403433207421290ba75c2fc084982" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "581403433207421290ba75c2fc084982" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36733 } } }
I20260810 06:45:23.958199 20440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982 [sys.catalog]: This master's current role is: LEADER
W20260810 06:45:23.959729 20457 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:45:23.959791 20457 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:45:23.959874 20443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:23.960060 20443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:23.961644 20443 catalog_manager.cc:1347] Generated new cluster ID: 9013dc9a01054bb4a1c28dd8bd161cec
I20260810 06:45:23.961706 20443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:45:23 dist-test-slave-vxj2 krb5kdc[20339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:23.967274 20437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:23.954655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37100 (local address 127.15.232.254:36733)
0810 06:45:23.954858 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:23.954869 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:23.954924 (+ 55us) server_negotiation.cc:407] Connection header received
0810 06:45:23.955825 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:23.955836 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:23.955897 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:23.956076 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:23.957009 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:23.958609 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:23.959562 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:23.960264 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:23.960412 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:23.963354 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:23.963359 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:23.963359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:23.963375 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:23.965374 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:23.965816 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:23.965823 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:23.965824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:23.965929 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:23.966162 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:23.966164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:23.966164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:23.966427 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:23.966782 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:23.966906 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 06:45:23.967067 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260810 06:45:23.980841 20443 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:23.981488 20443 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:23.984588 20443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 581403433207421290ba75c2fc084982: Generated new TSK 0
I20260810 06:45:23.984822 20443 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:45:24.021911 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:36733
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:24.121974 20464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:24.122186 20464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:24.122210 20464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:24.123308 20464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:24.123349 20464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:24.123376 20464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:24.124814 20464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:36733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:24.125006 20464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:24.125248 20464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:24.130323 20470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:24.130349 20469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:24.130475 20472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:24.130648 20464 server_base.cc:1034] running on GCE node
I20260810 06:45:24.130901 20464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:24.131163 20464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:24.132341 20464 hybrid_clock.cc:648] HybridClock initialized: now 1786344324132340 us; error 45 us; skew 500 ppm
Aug 10 06:45:24 dist-test-slave-vxj2 krb5kdc[20339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:24.135412 20464 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:45:24.135891 20464 webserver.cc:460] Webserver started at http://127.15.232.193:40867/ using document root <none> and password file <none>
I20260810 06:45:24.136140 20464 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:24.136215 20464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:24.136387 20464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:24.137697 20464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "11eb563482b547b694814680b7a8dd03"
format_stamp: "Formatted at 2026-08-10 06:45:24 on dist-test-slave-vxj2"
server_key: "e902f1a319b98c62130263cebd3e337e"
server_key_iv: "2d1781379142009ea217e4e9ecac48dd"
server_key_version: "encryptionkey@0"
I20260810 06:45:24.138103 20464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "11eb563482b547b694814680b7a8dd03"
format_stamp: "Formatted at 2026-08-10 06:45:24 on dist-test-slave-vxj2"
server_key: "e902f1a319b98c62130263cebd3e337e"
server_key_iv: "2d1781379142009ea217e4e9ecac48dd"
server_key_version: "encryptionkey@0"
I20260810 06:45:24.139571 20464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:24.140545 20481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:24.140744 20464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:24.140897 20464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "11eb563482b547b694814680b7a8dd03"
format_stamp: "Formatted at 2026-08-10 06:45:24 on dist-test-slave-vxj2"
server_key: "e902f1a319b98c62130263cebd3e337e"
server_key_iv: "2d1781379142009ea217e4e9ecac48dd"
server_key_version: "encryptionkey@0"
I20260810 06:45:24.140988 20464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:24.158358 20464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:24.161197 20464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:24.161393 20464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:24.161680 20464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:24.162046 20464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:24.162109 20464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:24.162155 20464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:24.162186 20464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:24.169066 20464 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:36755
I20260810 06:45:24.169126 20607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:36755 every 8 connection(s)
I20260810 06:45:24.169515 20464 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
Aug 10 06:45:24 dist-test-slave-vxj2 krb5kdc[20339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:24.178123 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 20464
I20260810 06:45:24.178293 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:24.178500 16291 external_mini_cluster.cc:1428] Setting key c328db893393a648392849e497141954
I20260810 06:45:24.180505 20611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:24.170296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:36733 (local address 127.15.232.193:44217)
0810 06:45:24.170501 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:24.170545 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:24.171500 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:24.171757 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:24.171760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:24.172037 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:24.172242 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:24.172246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.173078 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.173079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:24.174026 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:24.174029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.174470 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.174471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:24.174575 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:24.175125 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:24.175134 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:24.176678 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:24.178896 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:24.178905 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:24.178908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:24.179255 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:24.179493 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:24.179494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:24.179494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:24.179609 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:24.179945 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:24.179952 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:24.180088 (+ 136us) client_negotiation.cc:769] Sending connection context
0810 06:45:24.180190 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 06:45:24.180326 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":125,"threads_started":1}
I20260810 06:45:24.180497 20437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:24.170352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:44217 (local address 127.15.232.254:36733)
0810 06:45:24.170430 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:24.170432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:24.170638 (+ 206us) server_negotiation.cc:407] Connection header received
0810 06:45:24.171608 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:24.171609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:24.171625 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:24.171689 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:24.172363 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.172949 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.174133 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.174386 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.174455 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:24.176902 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:24.176903 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:24.176904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:24.176912 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:24.178795 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:24.179350 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:24.179352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:24.179352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:24.179424 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:24.179679 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:24.179680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:24.179680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:24.179833 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:24.179919 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:24.180234 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 06:45:24.180351 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:45:24.181322 20608 heartbeater.cc:347] Connected to a master server at 127.15.232.254:36733
I20260810 06:45:24.181445 20608 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:24.181638 20608 heartbeater.cc:511] Master 127.15.232.254:36733 requested a full tablet report, sending...
I20260810 06:45:24.182322 20393 ts_manager.cc:194] Registered new tserver with Master: 11eb563482b547b694814680b7a8dd03 (127.15.232.193:36755)
I20260810 06:45:24.183046 20393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:44217
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:45:24.208900 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:36733
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:24.293512 20614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:24.293807 20614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:24.293850 20614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:24.295009 20614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:24.295045 20614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:24.295073 20614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:24.296517 20614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:36733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:24.296705 20614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:24.296880 20614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:24.302088 20620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:24.302095 20622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:24.302186 20619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:24.302347 20614 server_base.cc:1034] running on GCE node
I20260810 06:45:24.302619 20614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:24.302894 20614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:24.304070 20614 hybrid_clock.cc:648] HybridClock initialized: now 1786344324304067 us; error 43 us; skew 500 ppm
Aug 10 06:45:24 dist-test-slave-vxj2 krb5kdc[20339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:24.307030 20614 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:45:24.307523 20614 webserver.cc:460] Webserver started at http://127.15.232.194:38625/ using document root <none> and password file <none>
I20260810 06:45:24.307794 20614 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:24.307873 20614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:24.308039 20614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:24.309296 20614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "38ab0ede61b64c8d943b8745fed48189"
format_stamp: "Formatted at 2026-08-10 06:45:24 on dist-test-slave-vxj2"
server_key: "cd8dac61b445a26ffb2f64aa1f68198c"
server_key_iv: "36eb595b2cd568a6f1cfd24c77368967"
server_key_version: "encryptionkey@0"
I20260810 06:45:24.309707 20614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "38ab0ede61b64c8d943b8745fed48189"
format_stamp: "Formatted at 2026-08-10 06:45:24 on dist-test-slave-vxj2"
server_key: "cd8dac61b445a26ffb2f64aa1f68198c"
server_key_iv: "36eb595b2cd568a6f1cfd24c77368967"
server_key_version: "encryptionkey@0"
I20260810 06:45:24.311259 20614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:24.312120 20629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:24.312307 20614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:24.312422 20614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "38ab0ede61b64c8d943b8745fed48189"
format_stamp: "Formatted at 2026-08-10 06:45:24 on dist-test-slave-vxj2"
server_key: "cd8dac61b445a26ffb2f64aa1f68198c"
server_key_iv: "36eb595b2cd568a6f1cfd24c77368967"
server_key_version: "encryptionkey@0"
I20260810 06:45:24.312515 20614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:24.322844 20614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:24.325397 20614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:24.325583 20614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:24.325847 20614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:24.326192 20614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:24.326258 20614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:24.326297 20614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:24.326333 20614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:24.332787 20614 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:42141
I20260810 06:45:24.332835 20758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:42141 every 8 connection(s)
I20260810 06:45:24.333254 20614 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:24.334460 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 20614
I20260810 06:45:24.334585 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:24.334839 16291 external_mini_cluster.cc:1428] Setting key e7a7864b9e6f8845d1054e80354233a6
Aug 10 06:45:24 dist-test-slave-vxj2 krb5kdc[20339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:24.343361 20437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:24.334042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:44853 (local address 127.15.232.254:36733)
0810 06:45:24.334103 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:24.334105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:24.334391 (+ 286us) server_negotiation.cc:407] Connection header received
0810 06:45:24.335517 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:24.335519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:24.335531 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:24.335583 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:24.336243 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.336752 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.337839 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.338045 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.338135 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:24.340353 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:24.340355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:24.340355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:24.340363 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:24.341909 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:24.342318 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:24.342319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:24.342319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:24.342401 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:24.342653 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:24.342654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:24.342654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:24.342770 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:24.342846 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:24.343175 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 06:45:24.343271 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:45:24.343547 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:24.334012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:36733 (local address 127.15.232.194:44853)
0810 06:45:24.334203 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:24.334277 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:24.335406 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:24.335651 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:24.335655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:24.335941 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:24.336148 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:24.336153 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.336852 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.336853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:24.337741 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:24.337744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.338145 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.338146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:24.338272 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:24.338902 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:24.338910 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:24.340254 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:24.342005 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:24.342012 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:24.342015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:24.342238 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:24.342472 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:24.342473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:24.342473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:24.342578 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:24.342836 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:24.342843 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:24.343029 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 06:45:24.343141 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 06:45:24.343301 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260810 06:45:24.344048 20759 heartbeater.cc:347] Connected to a master server at 127.15.232.254:36733
I20260810 06:45:24.344168 20759 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
I20260810 06:45:24.344352 20759 heartbeater.cc:511] Master 127.15.232.254:36733 requested a full tablet report, sending...
I20260810 06:45:24.344764 20393 ts_manager.cc:194] Registered new tserver with Master: 38ab0ede61b64c8d943b8745fed48189 (127.15.232.194:42141)
I20260810 06:45:24.345227 20393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:44853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:45:24.364293 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:36733
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:24.457449 20764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:24.457785 20764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:24.457845 20764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:24.459601 20764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:24.459652 20764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:24.459682 20764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:24.461503 20764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:36733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:24.461777 20764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:24.462041 20764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:24.467557 20770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:24.467583 20774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:24.467847 20769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:24.467872 20764 server_base.cc:1034] running on GCE node
I20260810 06:45:24.468130 20764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:24.468374 20764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:24.469528 20764 hybrid_clock.cc:648] HybridClock initialized: now 1786344324469516 us; error 33 us; skew 500 ppm
Aug 10 06:45:24 dist-test-slave-vxj2 krb5kdc[20339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:24.472249 20764 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:45:24.472710 20764 webserver.cc:460] Webserver started at http://127.15.232.195:39403/ using document root <none> and password file <none>
I20260810 06:45:24.472971 20764 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:24.473043 20764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:24.473253 20764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:24.474368 20764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "d193b1ebc435471e9e54b3953b748d6d"
format_stamp: "Formatted at 2026-08-10 06:45:24 on dist-test-slave-vxj2"
server_key: "8c394b64071c9cd16d1d19521268bca0"
server_key_iv: "8f7ff80b986879035731b9f2309fc203"
server_key_version: "encryptionkey@0"
I20260810 06:45:24.474740 20764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "d193b1ebc435471e9e54b3953b748d6d"
format_stamp: "Formatted at 2026-08-10 06:45:24 on dist-test-slave-vxj2"
server_key: "8c394b64071c9cd16d1d19521268bca0"
server_key_iv: "8f7ff80b986879035731b9f2309fc203"
server_key_version: "encryptionkey@0"
I20260810 06:45:24.476162 20764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:24.477046 20782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:24.477362 20764 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:24.477483 20764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "d193b1ebc435471e9e54b3953b748d6d"
format_stamp: "Formatted at 2026-08-10 06:45:24 on dist-test-slave-vxj2"
server_key: "8c394b64071c9cd16d1d19521268bca0"
server_key_iv: "8f7ff80b986879035731b9f2309fc203"
server_key_version: "encryptionkey@0"
I20260810 06:45:24.477592 20764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:24.502090 20764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:24.504293 20764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:24.504469 20764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:24.504756 20764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:24.505209 20764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:24.505273 20764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:24.505317 20764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:24.505347 20764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:24.512297 20764 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:42453
I20260810 06:45:24.512344 20909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:42453 every 8 connection(s)
I20260810 06:45:24.512748 20764 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:45:24 dist-test-slave-vxj2 krb5kdc[20339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:24.520453 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 20764
I20260810 06:45:24.520587 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:24.520828 16291 external_mini_cluster.cc:1428] Setting key a613614e2d36b6fb473733783842968a
I20260810 06:45:24.523907 20437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:24.513621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:49381 (local address 127.15.232.254:36733)
0810 06:45:24.513691 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:24.513694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:24.513910 (+ 216us) server_negotiation.cc:407] Connection header received
0810 06:45:24.515032 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:24.515033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:24.515047 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:24.515116 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:24.515916 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.516472 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.517684 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.517901 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.517996 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:24.520546 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:24.520547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:24.520548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:24.520556 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:24.522368 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:24.522829 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:24.522831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:24.522831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:24.522900 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:24.523187 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:24.523188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:24.523188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:24.523322 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:24.523401 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:24.523711 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 06:45:24.523810 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:45:24.524132 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:24.513574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:36733 (local address 127.15.232.195:49381)
0810 06:45:24.513758 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:24.513814 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:24.514898 (+ 1084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:24.515195 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:24.515200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:24.515509 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:24.515807 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:24.515813 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.516580 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.516581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:24.517587 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:24.517590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.518008 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.518009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:24.518148 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:24.518681 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:24.518690 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:24.520423 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:24.522477 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:24.522486 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:24.522488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:24.522740 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:24.522983 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:24.522984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:24.522984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:24.523103 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:24.523405 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:24.523413 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:24.523564 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 06:45:24.523685 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 06:45:24.523861 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
I20260810 06:45:24.524628 20910 heartbeater.cc:347] Connected to a master server at 127.15.232.254:36733
I20260810 06:45:24.524742 20910 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:24.524941 20910 heartbeater.cc:511] Master 127.15.232.254:36733 requested a full tablet report, sending...
I20260810 06:45:24.525391 20393 ts_manager.cc:194] Registered new tserver with Master: d193b1ebc435471e9e54b3953b748d6d (127.15.232.195:42453)
I20260810 06:45:24.525722 20393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:49381
I20260810 06:45:24.532826 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:45:24 dist-test-slave-vxj2 krb5kdc[20339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:45:24 dist-test-slave-vxj2 krb5kdc[20339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.193@KRBTEST.COM
I20260810 06:45:24.557694 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:24.549405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49458 (local address 127.15.232.193:36755)
0810 06:45:24.549603 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:24.549609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:24.549640 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:45:24.549669 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:24.549671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:24.549688 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:24.549760 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:24.550323 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.551192 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.551936 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.552400 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.552472 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:24.554861 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:24.554868 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:24.554869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:24.554876 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:24.556360 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:24.556698 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:24.556702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:24.556702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:24.556767 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:24.556973 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:24.556974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:24.556975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:24.557142 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:24.557404 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:24.557469 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 06:45:24.557592 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
Aug 10 06:45:24 dist-test-slave-vxj2 krb5kdc[20339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:24.566375 20437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:24.559356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41390 (local address 127.15.232.254:36733)
0810 06:45:24.559435 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:24.559437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:24.559506 (+ 69us) server_negotiation.cc:407] Connection header received
0810 06:45:24.559584 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:24.559585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:24.559595 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:24.559641 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:24.560123 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.560620 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.561354 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:24.561563 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:24.561647 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:24.563962 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:24.563963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:24.563963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:24.563970 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:24.565358 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:24.565702 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:24.565703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:24.565703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:24.565752 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:24.565929 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:24.565930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:24.565930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:24.566066 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:24.566141 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:24.566251 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 06:45:24.566308 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260810 06:45:24.566691 20393 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41390
I20260810 06:45:24.567116 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 20464
I20260810 06:45:24.574404 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 20614
I20260810 06:45:24.581202 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 20764
I20260810 06:45:24.587517 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 20353
2026-08-10T06:45:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2038 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:24 dist-test-slave-vxj2 krb5kdc[20923](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:24 dist-test-slave-vxj2 krb5kdc[20923](info): set up 2 sockets
Aug 10 06:45:24 dist-test-slave-vxj2 krb5kdc[20923](info): commencing operation
krb5kdc: starting...
W20260810 06:45:25.742789 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:25 dist-test-slave-vxj2 krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:45:25.864650 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:35653
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:41889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:35653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:25.948676 20936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:25.948948 20936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:25.949000 20936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:25.950296 20936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:25.950372 20936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:25.950395 20936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:25.950415 20936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:25.950453 20936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:25.952018 20936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:41889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:35653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:35653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:25.952322 20936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:25.952569 20936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:25.957671 20941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:25.957674 20942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:25.957671 20944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:25.957919 20936 server_base.cc:1034] running on GCE node
I20260810 06:45:25.958294 20936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:25.958597 20936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:25.959797 20936 hybrid_clock.cc:648] HybridClock initialized: now 1786344325959771 us; error 41 us; skew 500 ppm
Aug 10 06:45:25 dist-test-slave-vxj2 krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344325, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:25.962503 20936 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:45:25.962898 20936 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:25.962976 20936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:25.963131 20936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:25.964229 20936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "1264f664767f4b43b894a66c9498cbe9"
format_stamp: "Formatted at 2026-08-10 06:45:25 on dist-test-slave-vxj2"
server_key: "23c8d534921dd3221b4c2de3fe92dd15"
server_key_iv: "0ca5a4a0ad6e9a676e7f4889185295b9"
server_key_version: "encryptionkey@0"
I20260810 06:45:25.964589 20936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "1264f664767f4b43b894a66c9498cbe9"
format_stamp: "Formatted at 2026-08-10 06:45:25 on dist-test-slave-vxj2"
server_key: "23c8d534921dd3221b4c2de3fe92dd15"
server_key_iv: "0ca5a4a0ad6e9a676e7f4889185295b9"
server_key_version: "encryptionkey@0"
I20260810 06:45:25.966059 20936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:25.966866 20951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:25.967061 20936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:25.967173 20936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "1264f664767f4b43b894a66c9498cbe9"
format_stamp: "Formatted at 2026-08-10 06:45:25 on dist-test-slave-vxj2"
server_key: "23c8d534921dd3221b4c2de3fe92dd15"
server_key_iv: "0ca5a4a0ad6e9a676e7f4889185295b9"
server_key_version: "encryptionkey@0"
I20260810 06:45:25.967247 20936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:25.992519 20936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:25.994733 20936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:25.994939 20936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:25.999104 20936 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:35653
I20260810 06:45:25.999135 21012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:35653 every 8 connection(s)
I20260810 06:45:25.999437 20936 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:26.000197 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 20936
I20260810 06:45:26.000353 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:26.000572 16291 external_mini_cluster.cc:1428] Setting key 09e2ff1eb837f908316607c9d4b8f73f
I20260810 06:45:26.000808 21013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:26.004495 21013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9: Bootstrap starting.
I20260810 06:45:26.005218 21013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:26.005600 21013 log.cc:824] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:26.006662 21013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9: No bootstrap required, opened a new log
Aug 10 06:45:26 dist-test-slave-vxj2 krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:26.008836 21013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1264f664767f4b43b894a66c9498cbe9" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35653 } }
I20260810 06:45:26.009019 21013 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:26.009047 21013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1264f664767f4b43b894a66c9498cbe9, State: Initialized, Role: FOLLOWER
I20260810 06:45:26.009251 21013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1264f664767f4b43b894a66c9498cbe9" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35653 } }
I20260810 06:45:26.009346 21013 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:26.009371 21013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:26.009439 21013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:26.010200 21013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1264f664767f4b43b894a66c9498cbe9" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35653 } }
I20260810 06:45:26.010311 21013 leader_election.cc:302] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1264f664767f4b43b894a66c9498cbe9; no voters:
I20260810 06:45:26.010493 21013 leader_election.cc:288] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:26.010552 21018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:26.010702 21018 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [term 1 LEADER]: Becoming Leader. State: Replica: 1264f664767f4b43b894a66c9498cbe9, State: Running, Role: LEADER
I20260810 06:45:26.010795 21013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:26.010797 21018 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1264f664767f4b43b894a66c9498cbe9" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35653 } }
I20260810 06:45:26.012930 21019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1264f664767f4b43b894a66c9498cbe9. Latest consensus state: current_term: 1 leader_uuid: "1264f664767f4b43b894a66c9498cbe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1264f664767f4b43b894a66c9498cbe9" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35653 } } }
I20260810 06:45:26.013101 21020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1264f664767f4b43b894a66c9498cbe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1264f664767f4b43b894a66c9498cbe9" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35653 } } }
I20260810 06:45:26.013363 21019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:26.013377 21020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:26.013602 21036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:26.014034 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.001229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37362 (local address 127.15.232.254:35653)
0810 06:45:26.001417 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.001428 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.001464 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:45:26.002098 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.002109 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.002175 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.002346 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:26.002966 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.004385 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.005446 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.006177 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.006320 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.009229 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:26.009234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:26.009235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:26.009254 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:26.011623 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:26.012082 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:26.012086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:26.012086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:26.012213 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:26.012482 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:26.012483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:26.012483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:26.012827 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:26.013097 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.013566 (+ 469us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.013811 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260810 06:45:26.014103 21036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:26.016170 21036 catalog_manager.cc:1347] Generated new cluster ID: 9b591ae704c2460bbb81562beacf4aef
I20260810 06:45:26.016219 21036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:26.021178 21036 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:26.021694 21036 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:26.033123 21036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9: Generated new TSK 0
I20260810 06:45:26.033390 21036 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:45:26.065495 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35653
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:41889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:26.149920 21041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:26.150203 21041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:26.150230 21041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:26.151419 21041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:26.151463 21041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:26.151491 21041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:26.153012 21041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:41889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:26.153323 21041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:26.153561 21041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:26.158927 21048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:26.158931 21047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:26.159065 21041 server_base.cc:1034] running on GCE node
W20260810 06:45:26.159241 21050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:26.159523 21041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:26.159737 21041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:26.160895 21041 hybrid_clock.cc:648] HybridClock initialized: now 1786344326160863 us; error 49 us; skew 500 ppm
Aug 10 06:45:26 dist-test-slave-vxj2 krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:26.163939 21041 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:45:26.164304 21041 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:26.164363 21041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:26.164505 21041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:26.165719 21041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "b34fa24c10854c1f88827ea31a6e17b1"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "1998a3ac15015e3824f134e651037ea9"
server_key_iv: "2b5811dd192c7edb89376402fd4c6438"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.166106 21041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "b34fa24c10854c1f88827ea31a6e17b1"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "1998a3ac15015e3824f134e651037ea9"
server_key_iv: "2b5811dd192c7edb89376402fd4c6438"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.167573 21041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:45:26.168541 21056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.168777 21041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:26.168916 21041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "b34fa24c10854c1f88827ea31a6e17b1"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "1998a3ac15015e3824f134e651037ea9"
server_key_iv: "2b5811dd192c7edb89376402fd4c6438"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.169003 21041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:26.190515 21041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:26.192780 21041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:26.192979 21041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:26.193339 21041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:26.193722 21041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:26.193794 21041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.193833 21041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:26.193883 21041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.200215 21041 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:38877
I20260810 06:45:26.200251 21180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:38877 every 8 connection(s)
I20260810 06:45:26.200570 21041 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:26.200906 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 21041
I20260810 06:45:26.201027 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:26.201315 16291 external_mini_cluster.cc:1428] Setting key 33b289863f2b74120edb1ecc7b295483
Aug 10 06:45:26 dist-test-slave-vxj2 krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
I20260810 06:45:26.212278 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.201701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:32815 (local address 127.15.232.254:35653)
0810 06:45:26.201804 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.201807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.202032 (+ 225us) server_negotiation.cc:407] Connection header received
0810 06:45:26.202697 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.202698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.202712 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.202768 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:26.203890 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.204475 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.205658 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.205915 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.206016 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.208792 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:26.208794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:26.208794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:26.208802 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:26.210639 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:26.211135 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:26.211136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:26.211137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:26.211192 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:26.211571 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:26.211573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:26.211573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:26.211709 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:26.211770 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.212130 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.212208 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:45:26.212680 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.201661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35653 (local address 127.15.232.193:32815)
0810 06:45:26.201866 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:26.201921 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:26.202580 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:26.202866 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:26.202871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:26.203194 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:26.203770 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.203777 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.204596 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.204598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.205549 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.205552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.206030 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.206031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.206181 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.206888 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:26.206897 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:26.208681 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:26.210733 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:26.210739 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:26.210741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:26.211063 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:26.211282 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:26.211283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:26.211283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:26.211517 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:26.211801 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:26.211809 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:26.212003 (+ 194us) client_negotiation.cc:769] Sending connection context
0810 06:45:26.212116 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 06:45:26.212264 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
I20260810 06:45:26.213310 21181 heartbeater.cc:347] Connected to a master server at 127.15.232.254:35653
I20260810 06:45:26.213445 21181 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:26.213624 21181 heartbeater.cc:511] Master 127.15.232.254:35653 requested a full tablet report, sending...
I20260810 06:45:26.214332 20971 ts_manager.cc:194] Registered new tserver with Master: b34fa24c10854c1f88827ea31a6e17b1 (127.15.232.193:38877)
I20260810 06:45:26.215349 20971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:32815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:45:26.228502 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35653
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:41889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:26.313859 21187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:26.314095 21187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:26.314121 21187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:26.315362 21187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:26.315425 21187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:26.315455 21187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:26.317019 21187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:41889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:26.317319 21187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:26.317565 21187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:26.323113 21196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:26.323215 21187 server_base.cc:1034] running on GCE node
W20260810 06:45:26.323316 21195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:26.323122 21198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:26.323583 21187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:26.323879 21187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:26.325081 21187 hybrid_clock.cc:648] HybridClock initialized: now 1786344326325051 us; error 57 us; skew 500 ppm
Aug 10 06:45:26 dist-test-slave-vxj2 krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:26.328017 21187 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:45:26.328428 21187 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:26.328485 21187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:26.328588 21187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:26.329807 21187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "fb91b3dd951e4247ab73534dc5d47d21"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "07af6b4fed4462e6577cb88ecc3e8fcd"
server_key_iv: "ceffbdc6be87facd486a167bdada0c99"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.330149 21187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "fb91b3dd951e4247ab73534dc5d47d21"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "07af6b4fed4462e6577cb88ecc3e8fcd"
server_key_iv: "ceffbdc6be87facd486a167bdada0c99"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.331542 21187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:26.332386 21205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.332587 21187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:26.332680 21187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "fb91b3dd951e4247ab73534dc5d47d21"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "07af6b4fed4462e6577cb88ecc3e8fcd"
server_key_iv: "ceffbdc6be87facd486a167bdada0c99"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.332737 21187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:26.362490 21187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:26.365358 21187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:26.365500 21187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:26.365743 21187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:26.366063 21187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:26.366099 21187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.366123 21187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:26.366138 21187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.373101 21187 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:35273
I20260810 06:45:26.373222 21335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:35273 every 8 connection(s)
I20260810 06:45:26.373580 21187 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:26.374677 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 21187
I20260810 06:45:26.374800 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:26.375051 16291 external_mini_cluster.cc:1428] Setting key 2d854165c76e48cc7d5692a4e614a5e7
Aug 10 06:45:26 dist-test-slave-vxj2 krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
I20260810 06:45:26.385187 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.374394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:36813 (local address 127.15.232.254:35653)
0810 06:45:26.374477 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.374479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.374784 (+ 305us) server_negotiation.cc:407] Connection header received
0810 06:45:26.375530 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.375532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.375545 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.375623 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:26.377256 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.377837 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.378911 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.379111 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.379202 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.382019 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:26.382021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:26.382021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:26.382030 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:26.383550 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:26.384068 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:26.384069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:26.384069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:26.384133 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:26.384405 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:26.384406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:26.384407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:26.384536 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:26.384610 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.384966 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.385055 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:45:26.385448 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.374375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35653 (local address 127.15.232.194:36813)
0810 06:45:26.374613 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:26.374670 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:26.375403 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:26.375885 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:26.375891 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:26.376343 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:26.377124 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.377134 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.377950 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.377952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.378805 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.378807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.379215 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.379216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.379354 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.379931 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:26.379940 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:26.381899 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:26.383662 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:26.383671 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:26.383673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:26.383981 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:26.384210 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:26.384211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:26.384211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:26.384334 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:26.384612 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:26.384620 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:26.384780 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 06:45:26.384941 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 06:45:26.385145 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":121,"threads_started":1}
I20260810 06:45:26.386140 21336 heartbeater.cc:347] Connected to a master server at 127.15.232.254:35653
I20260810 06:45:26.386265 21336 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:26.386492 21336 heartbeater.cc:511] Master 127.15.232.254:35653 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
I20260810 06:45:26.387036 20971 ts_manager.cc:194] Registered new tserver with Master: fb91b3dd951e4247ab73534dc5d47d21 (127.15.232.194:35273)
I20260810 06:45:26.387506 20971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:36813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:45:26.405357 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35653
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:41889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:26.489871 21342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:26.490150 21342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:26.490193 21342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:26.491422 21342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:26.491503 21342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:26.491554 21342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:26.493144 21342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:41889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:26.493484 21342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:26.493727 21342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:26.499285 21348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:26.499274 21349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:26.499274 21352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:26.499516 21342 server_base.cc:1034] running on GCE node
I20260810 06:45:26.499768 21342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:26.500001 21342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:26.501204 21342 hybrid_clock.cc:648] HybridClock initialized: now 1786344326501179 us; error 48 us; skew 500 ppm
Aug 10 06:45:26 dist-test-slave-vxj2 krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:26.503923 21342 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:45:26.504333 21342 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:26.504410 21342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:26.504570 21342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:26.505772 21342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "6e9bae2b9c98431fbb7a2cbe7178288e"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "3eac6e2045cd7e8ecf2bdc31910beb25"
server_key_iv: "5a92050c675e216d5d26482b598fb54b"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.506207 21342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "6e9bae2b9c98431fbb7a2cbe7178288e"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "3eac6e2045cd7e8ecf2bdc31910beb25"
server_key_iv: "5a92050c675e216d5d26482b598fb54b"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.507611 21342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:26.508450 21358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.508656 21342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:26.508769 21342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "6e9bae2b9c98431fbb7a2cbe7178288e"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "3eac6e2045cd7e8ecf2bdc31910beb25"
server_key_iv: "5a92050c675e216d5d26482b598fb54b"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.508849 21342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:26.530022 21342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:26.532343 21342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:26.532539 21342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:26.532814 21342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:26.533308 21342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:26.533357 21342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.533411 21342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:26.533437 21342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.539839 21342 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:34969
I20260810 06:45:26.539897 21488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:34969 every 8 connection(s)
I20260810 06:45:26.540200 21342 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:26.540925 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 21342
I20260810 06:45:26.541045 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:26.541277 16291 external_mini_cluster.cc:1428] Setting key 1486440a6fe754a4e501f61bbb21c10f
Aug 10 06:45:26 dist-test-slave-vxj2 krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:26.550047 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.541020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:36009 (local address 127.15.232.254:35653)
0810 06:45:26.541097 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.541099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.541384 (+ 285us) server_negotiation.cc:407] Connection header received
0810 06:45:26.541961 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.541962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.541974 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.542040 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:26.543083 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.543584 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.544546 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.544728 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.544780 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.547006 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:26.547008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:26.547008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:26.547015 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:26.548622 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:26.548994 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:26.548995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:26.548995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:26.549053 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:26.549379 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:26.549380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:26.549380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:26.549522 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:26.549608 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.549903 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.549974 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:45:26.550163 21492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.540987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35653 (local address 127.15.232.195:36009)
0810 06:45:26.541236 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:26.541287 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:26.541886 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:26.542132 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:26.542136 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:26.542479 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:26.542990 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.542995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.543672 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.543673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.544472 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.544473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.544804 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.544805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.544908 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.545461 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:26.545470 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:26.546894 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:26.548697 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:26.548703 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:26.548705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:26.548921 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:26.549115 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:26.549116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:26.549116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:26.549287 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:26.549608 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:26.549613 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:26.549742 (+ 129us) client_negotiation.cc:769] Sending connection context
0810 06:45:26.549826 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 06:45:26.549957 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":117,"threads_started":1}
I20260810 06:45:26.550740 21489 heartbeater.cc:347] Connected to a master server at 127.15.232.254:35653
I20260810 06:45:26.550832 21489 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:26.551005 21489 heartbeater.cc:511] Master 127.15.232.254:35653 requested a full tablet report, sending...
I20260810 06:45:26.551355 20971 ts_manager.cc:194] Registered new tserver with Master: 6e9bae2b9c98431fbb7a2cbe7178288e (127.15.232.195:34969)
I20260810 06:45:26.551749 20971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:36009
I20260810 06:45:26.553105 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:45:26.560495 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.554469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37370 (local address 127.15.232.254:35653)
0810 06:45:26.554530 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.554532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.554629 (+ 97us) server_negotiation.cc:407] Connection header received
0810 06:45:26.554666 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.554667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.554675 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.554723 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:26.555252 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.555711 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.556484 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.556665 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.556721 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.558008 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:26.558009 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:26.558009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:26.558015 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:26.559468 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:26.559818 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:26.559819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:26.559819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:26.559877 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:26.560046 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:26.560047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:26.560047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:26.560193 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:26.560274 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.560356 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.560421 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 06:45:26.562376 20971 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:26.563966 20971 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:26.570648 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.566720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42964 (local address 127.15.232.194:35273)
0810 06:45:26.566930 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.566934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.566968 (+ 34us) server_negotiation.cc:407] Connection header received
0810 06:45:26.567026 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.567027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.567040 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.567143 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:26.567331 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.568205 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.569482 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.570242 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.570327 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.570386 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.570521 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.570559 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260810 06:45:26.570637 21503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.566431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:35273 (local address 127.0.0.1:42964)
0810 06:45:26.566914 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:26.566923 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:26.566974 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:26.567168 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:26.567170 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:26.567173 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:26.567245 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.567247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.568301 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.568302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.569372 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.569375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.570323 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.570324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.570411 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.570426 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 06:45:26.570503 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 06:45:26.570559 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":1,"thread_start_us":124,"threads_started":1}
I20260810 06:45:26.571120 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.566417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:38877 (local address 127.0.0.1:59744)
0810 06:45:26.566586 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:26.566627 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:26.566695 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:26.567275 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:26.567276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:26.567278 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:26.567342 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.567344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.568653 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.568654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.569703 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.569721 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.570652 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.570653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.570932 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.570936 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:26.570985 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 06:45:26.571025 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":59,"threads_started":1}
I20260810 06:45:26.571231 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.566689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52774 (local address 127.15.232.195:34969)
0810 06:45:26.567431 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.567435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.567445 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:26.567484 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.567486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.567503 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.567590 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:26.567751 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.568816 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.570116 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.570831 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.570902 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.570955 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.571095 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.571139 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":89,"threads_started":1}
I20260810 06:45:26.571228 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.566418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:34969 (local address 127.0.0.1:52774)
0810 06:45:26.566751 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:26.566763 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:26.566819 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:26.567604 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:26.567605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:26.567607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:26.567674 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.567676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.568920 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.568921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.570022 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.570025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.570926 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.570927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.571006 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.571011 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:45:26.571067 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 06:45:26.571111 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":7,"thread_start_us":127,"threads_started":1}
I20260810 06:45:26.571609 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.566554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59744 (local address 127.15.232.193:38877)
0810 06:45:26.567072 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.567076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.567084 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:26.567116 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.567117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.567155 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.567276 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:26.567433 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.568532 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.569799 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.570483 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.570557 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.570614 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.571416 (+ 802us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.571516 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":78,"threads_started":1}
I20260810 06:45:26.571676 21261 tablet_service.cc:1467] Processing CreateTablet for tablet 32418f92156a44d4b6e8ac2884a763ce (DEFAULT_TABLE table=test-table [id=7c00a08d1ed04744913eed26f24f2a42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:26.572070 21261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32418f92156a44d4b6e8ac2884a763ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:26.572081 21411 tablet_service.cc:1467] Processing CreateTablet for tablet 32418f92156a44d4b6e8ac2884a763ce (DEFAULT_TABLE table=test-table [id=7c00a08d1ed04744913eed26f24f2a42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:26.572407 21411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32418f92156a44d4b6e8ac2884a763ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:26.573572 21106 tablet_service.cc:1467] Processing CreateTablet for tablet 32418f92156a44d4b6e8ac2884a763ce (DEFAULT_TABLE table=test-table [id=7c00a08d1ed04744913eed26f24f2a42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:26.573858 21106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32418f92156a44d4b6e8ac2884a763ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:26.575611 21507 tablet_bootstrap.cc:492] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21: Bootstrap starting.
I20260810 06:45:26.576366 21507 tablet_bootstrap.cc:654] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:26.577041 21507 log.cc:824] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:26.577246 21509 tablet_bootstrap.cc:492] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1: Bootstrap starting.
I20260810 06:45:26.577348 21508 tablet_bootstrap.cc:492] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e: Bootstrap starting.
I20260810 06:45:26.577977 21509 tablet_bootstrap.cc:654] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:26.577977 21508 tablet_bootstrap.cc:654] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:26.578200 21507 tablet_bootstrap.cc:492] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21: No bootstrap required, opened a new log
I20260810 06:45:26.578297 21507 ts_tablet_manager.cc:1397] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:45:26.578615 21509 log.cc:824] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:26.578615 21508 log.cc:824] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:26.579416 21508 tablet_bootstrap.cc:492] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e: No bootstrap required, opened a new log
I20260810 06:45:26.579490 21508 ts_tablet_manager.cc:1397] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:45:26.579666 21509 tablet_bootstrap.cc:492] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1: No bootstrap required, opened a new log
I20260810 06:45:26.579735 21509 ts_tablet_manager.cc:1397] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:45:26.580358 21507 raft_consensus.cc:348] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34fa24c10854c1f88827ea31a6e17b1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38877 } } peers { permanent_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 34969 } } peers { permanent_uuid: "fb91b3dd951e4247ab73534dc5d47d21" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 35273 } }
I20260810 06:45:26.580524 21507 raft_consensus.cc:374] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:26.580574 21507 raft_consensus.cc:729] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb91b3dd951e4247ab73534dc5d47d21, State: Initialized, Role: FOLLOWER
I20260810 06:45:26.580717 21507 consensus_queue.cc:260] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34fa24c10854c1f88827ea31a6e17b1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38877 } } peers { permanent_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 34969 } } peers { permanent_uuid: "fb91b3dd951e4247ab73534dc5d47d21" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 35273 } }
I20260810 06:45:26.581039 21507 ts_tablet_manager.cc:1428] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:26.581215 21336 heartbeater.cc:503] Master 127.15.232.254:35653 was elected leader, sending a full tablet report...
I20260810 06:45:26.581512 21508 raft_consensus.cc:348] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34fa24c10854c1f88827ea31a6e17b1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38877 } } peers { permanent_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 34969 } } peers { permanent_uuid: "fb91b3dd951e4247ab73534dc5d47d21" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 35273 } }
I20260810 06:45:26.581668 21508 raft_consensus.cc:374] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:26.581703 21508 raft_consensus.cc:729] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e9bae2b9c98431fbb7a2cbe7178288e, State: Initialized, Role: FOLLOWER
I20260810 06:45:26.581818 21508 consensus_queue.cc:260] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34fa24c10854c1f88827ea31a6e17b1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38877 } } peers { permanent_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 34969 } } peers { permanent_uuid: "fb91b3dd951e4247ab73534dc5d47d21" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 35273 } }
I20260810 06:45:26.582118 21489 heartbeater.cc:503] Master 127.15.232.254:35653 was elected leader, sending a full tablet report...
I20260810 06:45:26.582209 21509 raft_consensus.cc:348] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34fa24c10854c1f88827ea31a6e17b1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38877 } } peers { permanent_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 34969 } } peers { permanent_uuid: "fb91b3dd951e4247ab73534dc5d47d21" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 35273 } }
I20260810 06:45:26.582365 21509 raft_consensus.cc:374] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:26.582106 21508 ts_tablet_manager.cc:1428] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:26.582418 21509 raft_consensus.cc:729] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b34fa24c10854c1f88827ea31a6e17b1, State: Initialized, Role: FOLLOWER
I20260810 06:45:26.582535 21509 consensus_queue.cc:260] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34fa24c10854c1f88827ea31a6e17b1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38877 } } peers { permanent_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 34969 } } peers { permanent_uuid: "fb91b3dd951e4247ab73534dc5d47d21" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 35273 } }
I20260810 06:45:26.582873 21509 ts_tablet_manager.cc:1428] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:26.582930 21181 heartbeater.cc:503] Master 127.15.232.254:35653 was elected leader, sending a full tablet report...
W20260810 06:45:26.624253 21337 tablet.cc:2369] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:45:26.625425 21514 raft_consensus.cc:482] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:26.625607 21514 raft_consensus.cc:504] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34fa24c10854c1f88827ea31a6e17b1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38877 } } peers { permanent_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 34969 } } peers { permanent_uuid: "fb91b3dd951e4247ab73534dc5d47d21" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 35273 } }
I20260810 06:45:26.626091 21514 leader_election.cc:288] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b34fa24c10854c1f88827ea31a6e17b1 (127.15.232.193:38877), fb91b3dd951e4247ab73534dc5d47d21 (127.15.232.194:35273)
I20260810 06:45:26.629899 21492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.626270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:35273 (local address 127.15.232.195:51179)
0810 06:45:26.626340 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:26.626357 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:26.626425 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:26.626704 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:26.626706 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:26.626708 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:26.626803 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.626806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.627629 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.627630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.628898 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.628900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.629703 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.629704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.629767 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.629771 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:26.629811 (+ 40us) client_negotiation.cc:240] Negotiation successful
0810 06:45:26.629841 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 06:45:26.629971 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.626447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:51179 (local address 127.15.232.194:35273)
0810 06:45:26.626520 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.626523 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.626534 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:26.626565 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.626566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.626576 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.626641 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:26.626898 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.627525 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.629021 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.629572 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.629683 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.629695 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.629846 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.629893 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:45:26.630306 21282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32418f92156a44d4b6e8ac2884a763ce" candidate_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91b3dd951e4247ab73534dc5d47d21" is_pre_election: true
I20260810 06:45:26.630509 21282 raft_consensus.cc:2454] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e9bae2b9c98431fbb7a2cbe7178288e in term 0.
I20260810 06:45:26.630847 21361 leader_election.cc:302] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e9bae2b9c98431fbb7a2cbe7178288e, fb91b3dd951e4247ab73534dc5d47d21; no voters:
I20260810 06:45:26.630882 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.626882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:43619 (local address 127.15.232.193:38877)
0810 06:45:26.626983 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.626985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.627325 (+ 340us) server_negotiation.cc:407] Connection header received
0810 06:45:26.627465 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.627466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.627477 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.627514 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:26.627775 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.628494 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.629650 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.630369 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.630471 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.630481 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.630758 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.630809 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 06:45:26.630980 21516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.626840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:38877 (local address 127.15.232.195:43619)
0810 06:45:26.627239 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:26.627253 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:26.627360 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:26.627591 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:26.627592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:26.627594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:26.627675 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.627677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.628601 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.628602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.629555 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:26.629559 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.630585 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.630587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:26.630666 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.630671 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:45:26.630746 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:45:26.630845 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":135,"threads_started":1}
I20260810 06:45:26.631042 21514 raft_consensus.cc:2781] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:26.631102 21514 raft_consensus.cc:482] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:26.631150 21514 raft_consensus.cc:3037] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:26.631356 21127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32418f92156a44d4b6e8ac2884a763ce" candidate_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b34fa24c10854c1f88827ea31a6e17b1" is_pre_election: true
I20260810 06:45:26.631513 21127 raft_consensus.cc:2454] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e9bae2b9c98431fbb7a2cbe7178288e in term 0.
I20260810 06:45:26.632176 21514 raft_consensus.cc:504] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34fa24c10854c1f88827ea31a6e17b1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38877 } } peers { permanent_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 34969 } } peers { permanent_uuid: "fb91b3dd951e4247ab73534dc5d47d21" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 35273 } }
I20260810 06:45:26.632371 21514 leader_election.cc:288] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [CANDIDATE]: Term 1 election: Requested vote from peers b34fa24c10854c1f88827ea31a6e17b1 (127.15.232.193:38877), fb91b3dd951e4247ab73534dc5d47d21 (127.15.232.194:35273)
I20260810 06:45:26.632579 21127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32418f92156a44d4b6e8ac2884a763ce" candidate_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b34fa24c10854c1f88827ea31a6e17b1"
I20260810 06:45:26.632570 21282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32418f92156a44d4b6e8ac2884a763ce" candidate_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91b3dd951e4247ab73534dc5d47d21"
I20260810 06:45:26.632661 21127 raft_consensus.cc:3037] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:26.632686 21282 raft_consensus.cc:3037] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:26.633498 21127 raft_consensus.cc:2454] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e9bae2b9c98431fbb7a2cbe7178288e in term 1.
I20260810 06:45:26.633633 21282 raft_consensus.cc:2454] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e9bae2b9c98431fbb7a2cbe7178288e in term 1.
I20260810 06:45:26.633677 21359 leader_election.cc:302] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e9bae2b9c98431fbb7a2cbe7178288e, b34fa24c10854c1f88827ea31a6e17b1; no voters:
I20260810 06:45:26.633865 21514 raft_consensus.cc:2781] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:26.634083 21514 raft_consensus.cc:686] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 1 LEADER]: Becoming Leader. State: Replica: 6e9bae2b9c98431fbb7a2cbe7178288e, State: Running, Role: LEADER
I20260810 06:45:26.634203 21514 consensus_queue.cc:237] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34fa24c10854c1f88827ea31a6e17b1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38877 } } peers { permanent_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 34969 } } peers { permanent_uuid: "fb91b3dd951e4247ab73534dc5d47d21" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 35273 } }
I20260810 06:45:26.635105 20970 catalog_manager.cc:5495] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e9bae2b9c98431fbb7a2cbe7178288e (127.15.232.195). New cstate: current_term: 1 leader_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34fa24c10854c1f88827ea31a6e17b1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e9bae2b9c98431fbb7a2cbe7178288e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 34969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb91b3dd951e4247ab73534dc5d47d21" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 35273 } health_report { overall_health: UNKNOWN } } }
I20260810 06:45:26.644657 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.642138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52788 (local address 127.15.232.195:34969)
0810 06:45:26.642215 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.642218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.642226 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:26.642270 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.642271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.642293 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.642367 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:26.642525 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.643013 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.643834 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:26.644027 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:26.644106 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:26.644306 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:26.644420 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:26.644510 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.644552 (+ 42us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.644573 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:45:26.646605 21282 raft_consensus.cc:1261] T 32418f92156a44d4b6e8ac2884a763ce P fb91b3dd951e4247ab73534dc5d47d21 [term 1 FOLLOWER]: Refusing update from remote peer 6e9bae2b9c98431fbb7a2cbe7178288e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:26.646605 21127 raft_consensus.cc:1261] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1 [term 1 FOLLOWER]: Refusing update from remote peer 6e9bae2b9c98431fbb7a2cbe7178288e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:26.647045 21514 consensus_queue.cc:1035] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b34fa24c10854c1f88827ea31a6e17b1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:26.647207 21517 consensus_queue.cc:1035] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb91b3dd951e4247ab73534dc5d47d21" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:26.649385 21520 mvcc.cc:205] Tried to move back new op lower bound from 7316866361941671936 to 7316866361894535168. Current Snapshot: MvccSnapshot[applied={T|T < 7316866361941671936}]
I20260810 06:45:26.650046 21521 mvcc.cc:205] Tried to move back new op lower bound from 7316866361941671936 to 7316866361894535168. Current Snapshot: MvccSnapshot[applied={T|T < 7316866361941671936 or (T in {7316866361941671936})}]
I20260810 06:45:26.650470 21522 mvcc.cc:205] Tried to move back new op lower bound from 7316866361941671936 to 7316866361894535168. Current Snapshot: MvccSnapshot[applied={T|T < 7316866361941671936}]
I20260810 06:45:26.651767 20970 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:26.651861 20970 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:26.652577 20970 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1264f664767f4b43b894a66c9498cbe9: Sending DeleteTablet for 3 replicas of tablet 32418f92156a44d4b6e8ac2884a763ce
I20260810 06:45:26.652943 21106 tablet_service.cc:1514] Processing DeleteTablet for tablet 32418f92156a44d4b6e8ac2884a763ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:26 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:59744
I20260810 06:45:26.653007 21411 tablet_service.cc:1514] Processing DeleteTablet for tablet 32418f92156a44d4b6e8ac2884a763ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:26 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:52774
I20260810 06:45:26.653193 21531 tablet_replica.cc:330] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1: stopping tablet replica
I20260810 06:45:26.653266 21532 tablet_replica.cc:330] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e: stopping tablet replica
I20260810 06:45:26.653294 21531 raft_consensus.cc:2229] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:26.653249 21261 tablet_service.cc:1514] Processing DeleteTablet for tablet 32418f92156a44d4b6e8ac2884a763ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:26 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:42964
I20260810 06:45:26.653342 21532 raft_consensus.cc:2229] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:26.653389 21531 raft_consensus.cc:2258] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:26.653426 21532 raft_consensus.cc:2258] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:26.653874 21531 ts_tablet_manager.cc:1905] T 32418f92156a44d4b6e8ac2884a763ce P b34fa24c10854c1f88827ea31a6e17b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:26.653874 21532 ts_tablet_manager.cc:1905] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:26.653940 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 21041
I20260810 06:45:26.656792 21532 ts_tablet_manager.cc:1918] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:26.656862 21532 log.cc:1196] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/32418f92156a44d4b6e8ac2884a763ce
I20260810 06:45:26.657097 21532 ts_tablet_manager.cc:1939] T 32418f92156a44d4b6e8ac2884a763ce P 6e9bae2b9c98431fbb7a2cbe7178288e: Deleting consensus metadata
I20260810 06:45:26.658188 20954 catalog_manager.cc:4841] TS 6e9bae2b9c98431fbb7a2cbe7178288e (127.15.232.195:34969): tablet 32418f92156a44d4b6e8ac2884a763ce (table test-table [id=7c00a08d1ed04744913eed26f24f2a42]) successfully deleted
W20260810 06:45:26.662012 20953 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:38877 (error 108)
I20260810 06:45:26.662225 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 21187
I20260810 06:45:26.665524 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.665055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:38877 (local address 127.0.0.1:59746)
0810 06:45:26.665431 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:26.665478 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:38877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353}
W20260810 06:45:26.668956 20953 catalog_manager.cc:4584] TS b34fa24c10854c1f88827ea31a6e17b1 (127.15.232.193:38877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 32418f92156a44d4b6e8ac2884a763ce: Network error: Client connection negotiation failed: client connection to 127.15.232.193:38877: connect: Connection refused (error 111)
I20260810 06:45:26.673305 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 21342
I20260810 06:45:26.673338 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.673210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:35273 (local address 127.0.0.1:42970)
0810 06:45:26.673272 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:26.673308 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.194:35273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 06:45:26.679742 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 20936
2026-08-10T06:45:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2093 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T06:45:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:26Z Disabled control of system clock
I20260810 06:45:26.712054 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:37085
--webserver_interface=127.15.232.254
--webserver_port=0
--builtin_ntp_servers=127.15.232.212:32939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:37085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 06:45:26.797000 21540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:26.797271 21540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:26.797298 21540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:26.798480 21540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:26.798530 21540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:26.798543 21540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:26.798594 21540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:26.798614 21540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:26.800014 21540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:32939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:37085
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:37085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:26.800285 21540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:26.800599 21540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:26.805498 21547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:26.805483 21549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:26.805473 21546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:26.805761 21540 server_base.cc:1034] running on GCE node
I20260810 06:45:26.806089 21540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:26.806396 21540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:26.807597 21540 hybrid_clock.cc:648] HybridClock initialized: now 1786344326807582 us; error 44 us; skew 500 ppm
I20260810 06:45:26.808905 21540 webserver.cc:460] Webserver started at http://127.15.232.254:35759/ using document root <none> and password file <none>
I20260810 06:45:26.809255 21540 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:26.809329 21540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:26.809491 21540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:26.810591 21540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "c3b8062ed60e4c9ab409c97030c785ae"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "97ab7956de135db84665cde4b52f96e6"
server_key_iv: "dd7eb3f7994d867a33d4fc0a0f7eca49"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.810994 21540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "c3b8062ed60e4c9ab409c97030c785ae"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "97ab7956de135db84665cde4b52f96e6"
server_key_iv: "dd7eb3f7994d867a33d4fc0a0f7eca49"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.812315 21540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:26.813028 21555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.813226 21540 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:26.813330 21540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "c3b8062ed60e4c9ab409c97030c785ae"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "97ab7956de135db84665cde4b52f96e6"
server_key_iv: "dd7eb3f7994d867a33d4fc0a0f7eca49"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.813416 21540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:26.816776 21540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:26.817075 21540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:26.817248 21540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:26.821056 21540 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:37085
I20260810 06:45:26.821087 21614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:37085 every 8 connection(s)
I20260810 06:45:26.821466 21540 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:26.822100 21615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:26.824862 21615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae: Bootstrap starting.
I20260810 06:45:26.825541 21615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:26.825845 21615 log.cc:824] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:26.826447 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 21540
I20260810 06:45:26.826552 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:26.826581 21615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae: No bootstrap required, opened a new log
I20260810 06:45:26.826750 16291 external_mini_cluster.cc:1428] Setting key bd81537cf43977926c4fe7ce9f05bccc
I20260810 06:45:26.828806 21615 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8062ed60e4c9ab409c97030c785ae" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37085 } }
I20260810 06:45:26.828989 21615 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:26.829030 21615 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b8062ed60e4c9ab409c97030c785ae, State: Initialized, Role: FOLLOWER
I20260810 06:45:26.829203 21615 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8062ed60e4c9ab409c97030c785ae" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37085 } }
I20260810 06:45:26.829308 21615 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:26.829351 21615 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:26.829414 21615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:26.830344 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.827313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54198 (local address 127.15.232.254:37085)
0810 06:45:26.827549 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.827559 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.827593 (+ 34us) server_negotiation.cc:407] Connection header received
0810 06:45:26.828755 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.828765 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.828819 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.828990 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:26.829837 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:26.829841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:26.829841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:26.829846 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:26.829916 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:26.829934 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:26.830030 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.830077 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.830145 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
I20260810 06:45:26.830497 21615 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8062ed60e4c9ab409c97030c785ae" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37085 } }
I20260810 06:45:26.830612 21615 leader_election.cc:302] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3b8062ed60e4c9ab409c97030c785ae; no voters:
I20260810 06:45:26.830796 21615 leader_election.cc:288] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:26.830825 21620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:26.830945 21620 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [term 1 LEADER]: Becoming Leader. State: Replica: c3b8062ed60e4c9ab409c97030c785ae, State: Running, Role: LEADER
I20260810 06:45:26.831058 21615 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:26.831039 21620 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8062ed60e4c9ab409c97030c785ae" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37085 } }
I20260810 06:45:26.831391 21621 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3b8062ed60e4c9ab409c97030c785ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8062ed60e4c9ab409c97030c785ae" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37085 } } }
I20260810 06:45:26.831491 21621 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:26.831408 21622 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b8062ed60e4c9ab409c97030c785ae. Latest consensus state: current_term: 1 leader_uuid: "c3b8062ed60e4c9ab409c97030c785ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8062ed60e4c9ab409c97030c785ae" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37085 } } }
I20260810 06:45:26.831650 21622 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:26.832047 21628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:26.832538 21628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:26.834147 21628 catalog_manager.cc:1347] Generated new cluster ID: 33044c1b14f342189dc2fc16b9281d1b
I20260810 06:45:26.834203 21628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:26.839818 21628 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:26.839999 21628 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:26.843128 21628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae: Generated new TSK 0
I20260810 06:45:26.843323 21628 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:45:26.849048 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:37085
--builtin_ntp_servers=127.15.232.212:32939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 06:45:26.938120 21644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:26.938370 21644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:26.938410 21644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:26.939623 21644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:26.939692 21644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:26.939739 21644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:26.941218 21644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:32939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:37085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:26.941496 21644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:26.941818 21644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:26.947052 21650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:26.947072 21652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:26.947052 21649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:26.947537 21644 server_base.cc:1034] running on GCE node
I20260810 06:45:26.947762 21644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:26.948007 21644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:26.949198 21644 hybrid_clock.cc:648] HybridClock initialized: now 1786344326949173 us; error 50 us; skew 500 ppm
I20260810 06:45:26.950534 21644 webserver.cc:460] Webserver started at http://127.15.232.193:35177/ using document root <none> and password file <none>
I20260810 06:45:26.950786 21644 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:26.950860 21644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:26.951004 21644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:26.952140 21644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "a3625350e8de4831a9026529b24dc901"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "e65378c75698a2c24e6737ca3023c0a0"
server_key_iv: "1164ccb32b6f48c39df2d2203ac5905e"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.952558 21644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "a3625350e8de4831a9026529b24dc901"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "e65378c75698a2c24e6737ca3023c0a0"
server_key_iv: "1164ccb32b6f48c39df2d2203ac5905e"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.954057 21644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:26.954885 21659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.955091 21644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:26.955179 21644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "a3625350e8de4831a9026529b24dc901"
format_stamp: "Formatted at 2026-08-10 06:45:26 on dist-test-slave-vxj2"
server_key: "e65378c75698a2c24e6737ca3023c0a0"
server_key_iv: "1164ccb32b6f48c39df2d2203ac5905e"
server_key_version: "encryptionkey@0"
I20260810 06:45:26.955271 21644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:26.958678 21644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:26.958966 21644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:26.959113 21644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:26.959335 21644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:26.959695 21644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:26.959750 21644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.959789 21644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:26.959815 21644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:26.967149 21644 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:36637
I20260810 06:45:26.967178 21784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:36637 every 8 connection(s)
I20260810 06:45:26.967552 21644 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:26.970806 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.968343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:59985 (local address 127.15.232.254:37085)
0810 06:45:26.968410 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:26.968412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:26.968670 (+ 258us) server_negotiation.cc:407] Connection header received
0810 06:45:26.969229 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:26.969231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:26.969238 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:26.969296 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:26.970546 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:26.970547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:26.970548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:26.970553 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:26.970568 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:26.970572 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:26.970626 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:26.970717 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 06:45:26.970734 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:45:26.971045 21788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:26.968329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:37085 (local address 127.15.232.193:59985)
0810 06:45:26.968540 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:26.968575 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:26.969088 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:26.969381 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:26.969385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:26.969393 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:26.970383 (+ 990us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:26.970383 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:26.970414 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:26.970442 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:26.970623 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:26.970627 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:26.970631 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:26.970728 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:45:26.970793 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260810 06:45:26.971669 21786 heartbeater.cc:347] Connected to a master server at 127.15.232.254:37085
I20260810 06:45:26.971755 21786 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:26.971930 21786 heartbeater.cc:511] Master 127.15.232.254:37085 requested a full tablet report, sending...
I20260810 06:45:26.972503 21573 ts_manager.cc:194] Registered new tserver with Master: a3625350e8de4831a9026529b24dc901 (127.15.232.193:36637)
I20260810 06:45:26.976652 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 21644
I20260810 06:45:26.976744 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:26.976884 16291 external_mini_cluster.cc:1428] Setting key cc7952ed7cb288e8644d1de01a09ea8a
I20260810 06:45:26.977794 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:37085
--builtin_ntp_servers=127.15.232.212:32939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 06:45:27.061133 21789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:27.061431 21789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:27.061456 21789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:27.062599 21789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:27.062633 21789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:27.062659 21789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:27.063944 21789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:32939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:37085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:27.064137 21789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:27.064385 21789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:27.069526 21797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:27.069578 21794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:27.069898 21795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:27.069887 21789 server_base.cc:1034] running on GCE node
I20260810 06:45:27.070178 21789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:27.070422 21789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:27.071604 21789 hybrid_clock.cc:648] HybridClock initialized: now 1786344327071586 us; error 27 us; skew 500 ppm
I20260810 06:45:27.073139 21789 webserver.cc:460] Webserver started at http://127.15.232.194:39311/ using document root <none> and password file <none>
I20260810 06:45:27.073494 21789 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:27.073567 21789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:27.073716 21789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:27.074889 21789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "1daaeb50df6a4644b213ce27fafbcdff"
format_stamp: "Formatted at 2026-08-10 06:45:27 on dist-test-slave-vxj2"
server_key: "b180f2606c77947bc91aae47203cea04"
server_key_iv: "eb6723959dc4176a96ee1709a50dae69"
server_key_version: "encryptionkey@0"
I20260810 06:45:27.075284 21789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "1daaeb50df6a4644b213ce27fafbcdff"
format_stamp: "Formatted at 2026-08-10 06:45:27 on dist-test-slave-vxj2"
server_key: "b180f2606c77947bc91aae47203cea04"
server_key_iv: "eb6723959dc4176a96ee1709a50dae69"
server_key_version: "encryptionkey@0"
I20260810 06:45:27.076785 21789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:27.077737 21805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:27.077996 21789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:45:27.078119 21789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "1daaeb50df6a4644b213ce27fafbcdff"
format_stamp: "Formatted at 2026-08-10 06:45:27 on dist-test-slave-vxj2"
server_key: "b180f2606c77947bc91aae47203cea04"
server_key_iv: "eb6723959dc4176a96ee1709a50dae69"
server_key_version: "encryptionkey@0"
I20260810 06:45:27.078213 21789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:27.082180 21789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:27.082535 21789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:27.082710 21789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:27.082939 21789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:27.083292 21789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:27.083360 21789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:27.083407 21789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:27.083447 21789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:27.090011 21789 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:37029
I20260810 06:45:27.090070 21927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:37029 every 8 connection(s)
I20260810 06:45:27.090443 21789 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:27.092782 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 21789
I20260810 06:45:27.092895 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:27.093122 16291 external_mini_cluster.cc:1428] Setting key 9baad84a465dbe51e330846d0a16c02e
I20260810 06:45:27.093822 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.091183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:45659 (local address 127.15.232.254:37085)
0810 06:45:27.091233 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.091235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.091493 (+ 258us) server_negotiation.cc:407] Connection header received
0810 06:45:27.091956 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.091957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.091964 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.092040 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:27.093541 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:27.093542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:27.093542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:27.093547 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:27.093561 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:27.093566 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:27.093642 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:27.093726 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 06:45:27.093742 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:45:27.094053 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.091125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:37085 (local address 127.15.232.194:45659)
0810 06:45:27.091328 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:27.091371 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:27.091858 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:27.092116 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:27.092120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:27.092128 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:27.093367 (+ 1239us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:27.093368 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:27.093410 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:27.093440 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:27.093643 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:27.093650 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:27.093653 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:27.093738 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:45:27.093804 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260810 06:45:27.094293 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:37085
--builtin_ntp_servers=127.15.232.212:32939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 06:45:27.094513 21928 heartbeater.cc:347] Connected to a master server at 127.15.232.254:37085
I20260810 06:45:27.094626 21928 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:27.094811 21928 heartbeater.cc:511] Master 127.15.232.254:37085 requested a full tablet report, sending...
I20260810 06:45:27.095170 21573 ts_manager.cc:194] Registered new tserver with Master: 1daaeb50df6a4644b213ce27fafbcdff (127.15.232.194:37029)
W20260810 06:45:27.183684 21932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:27.183931 21932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:27.183974 21932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:27.185137 21932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:27.185233 21932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:27.185322 21932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:27.186722 21932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:32939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:37085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:27.187002 21932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:27.187309 21932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:27.192571 21939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:27.192545 21940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:27.192679 21932 server_base.cc:1034] running on GCE node
W20260810 06:45:27.192544 21942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:27.192971 21932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:27.193259 21932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:27.194430 21932 hybrid_clock.cc:648] HybridClock initialized: now 1786344327194421 us; error 27 us; skew 500 ppm
I20260810 06:45:27.195667 21932 webserver.cc:460] Webserver started at http://127.15.232.195:46849/ using document root <none> and password file <none>
I20260810 06:45:27.195924 21932 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:27.195996 21932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:27.196151 21932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:27.197299 21932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "c75fb251ce5c4179bf9603b6f39323a6"
format_stamp: "Formatted at 2026-08-10 06:45:27 on dist-test-slave-vxj2"
server_key: "85511ce904b78eeb201407a6eb76878e"
server_key_iv: "473cc41310f7c56206c29f6cd06cea50"
server_key_version: "encryptionkey@0"
I20260810 06:45:27.197671 21932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "c75fb251ce5c4179bf9603b6f39323a6"
format_stamp: "Formatted at 2026-08-10 06:45:27 on dist-test-slave-vxj2"
server_key: "85511ce904b78eeb201407a6eb76878e"
server_key_iv: "473cc41310f7c56206c29f6cd06cea50"
server_key_version: "encryptionkey@0"
I20260810 06:45:27.199165 21932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:27.200156 21950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:27.200376 21932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:27.200510 21932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "c75fb251ce5c4179bf9603b6f39323a6"
format_stamp: "Formatted at 2026-08-10 06:45:27 on dist-test-slave-vxj2"
server_key: "85511ce904b78eeb201407a6eb76878e"
server_key_iv: "473cc41310f7c56206c29f6cd06cea50"
server_key_version: "encryptionkey@0"
I20260810 06:45:27.200598 21932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:27.204700 21932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:27.205008 21932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:27.205216 21932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:27.205452 21932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:27.205806 21932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:27.205868 21932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:27.205912 21932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:27.205943 21932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:27.212881 21932 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:33681
I20260810 06:45:27.212924 22077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:33681 every 8 connection(s)
I20260810 06:45:27.213379 21932 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:27.216528 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.214208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:38587 (local address 127.15.232.254:37085)
0810 06:45:27.214296 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.214299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.214533 (+ 234us) server_negotiation.cc:407] Connection header received
0810 06:45:27.214978 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.214979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.214983 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.215048 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:27.216313 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:27.216314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:27.216314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:27.216317 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:27.216329 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:27.216332 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:27.216379 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:27.216452 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 06:45:27.216467 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:45:27.216741 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.214171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:37085 (local address 127.15.232.195:38587)
0810 06:45:27.214396 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:27.214436 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:27.214898 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:27.215099 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:27.215102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:27.215109 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:27.216182 (+ 1073us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:27.216183 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:27.216211 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:27.216239 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:27.216377 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:27.216379 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:27.216381 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:45:27.216433 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:45:27.216481 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":101,"threads_started":1}
I20260810 06:45:27.217279 22078 heartbeater.cc:347] Connected to a master server at 127.15.232.254:37085
I20260810 06:45:27.217406 22078 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:27.217576 22078 heartbeater.cc:511] Master 127.15.232.254:37085 requested a full tablet report, sending...
I20260810 06:45:27.218021 21573 ts_manager.cc:194] Registered new tserver with Master: c75fb251ce5c4179bf9603b6f39323a6 (127.15.232.195:33681)
I20260810 06:45:27.220048 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 21932
I20260810 06:45:27.220129 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:27.220268 16291 external_mini_cluster.cc:1428] Setting key af7b36c32e9da4c10a3e2d8cc15cada4
I20260810 06:45:27.221143 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:45:27.223503 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.222456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54206 (local address 127.15.232.254:37085)
0810 06:45:27.222517 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.222520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.222611 (+ 91us) server_negotiation.cc:407] Connection header received
0810 06:45:27.222653 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.222653 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.222657 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.222707 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:27.223336 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:27.223337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:27.223337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:27.223340 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:27.223350 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:27.223353 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:27.223406 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:27.223437 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 06:45:27.223447 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:45:27.224978 21573 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:27.226668 21573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:27.230751 22087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.228858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:36637 (local address 127.0.0.1:35644)
0810 06:45:27.229203 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:27.229216 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:27.229268 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:27.229571 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:27.229578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:27.229585 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:27.230261 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:27.230261 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:27.230278 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:27.230301 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:27.230574 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:27.230580 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:27.230584 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:27.230657 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:45:27.230672 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260810 06:45:27.231035 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.229087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:37029 (local address 127.0.0.1:34100)
0810 06:45:27.229207 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:27.229216 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:27.229268 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:27.229747 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:27.229748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:27.229751 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:27.230668 (+ 917us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:27.230669 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:27.230676 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:27.230692 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:27.230935 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:27.230936 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:27.230936 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 06:45:27.230964 (+ 28us) client_negotiation.cc:240] Negotiation successful
0810 06:45:27.230976 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":201,"threads_started":1}
I20260810 06:45:27.231117 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.228857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:33681 (local address 127.0.0.1:49660)
0810 06:45:27.229047 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:27.229087 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:27.229142 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:27.230072 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:27.230074 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:27.230076 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:27.230637 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:27.230638 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:27.230645 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:27.230657 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:27.230899 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:27.230900 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:27.230900 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 06:45:27.230947 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 06:45:27.230958 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":120,"threads_started":1}
I20260810 06:45:27.231259 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.228988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.15.232.195:33681)
0810 06:45:27.229931 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.229935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.229943 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:27.229964 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.229965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.229971 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.230070 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:27.230733 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:27.230740 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:27.230740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:27.230744 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:27.230780 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:27.230802 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:27.230859 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:27.231077 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 06:45:27.231094 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":909,"thread_start_us":97,"threads_started":1}
I20260810 06:45:27.231359 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.229316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.15.232.194:37029)
0810 06:45:27.229584 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.229589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.229600 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:27.229629 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.229631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.229640 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.230235 (+ 595us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:27.230789 (+ 554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:27.230796 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:27.230797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:27.230801 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:27.230835 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:27.230861 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:27.231238 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:27.231261 (+ 23us) server_negotiation.cc:299] Negotiation successful
0810 06:45:27.231276 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":100,"threads_started":1}
I20260810 06:45:27.231071 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.228987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35644 (local address 127.15.232.193:36637)
0810 06:45:27.229410 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.229416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.229428 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:45:27.229457 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.229459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.229469 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.229553 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:27.230392 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:27.230407 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:27.230408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:27.230412 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:27.230472 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:27.230499 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:27.230580 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:27.230679 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 06:45:27.230698 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":100,"threads_started":1}
I20260810 06:45:27.232007 22002 tablet_service.cc:1467] Processing CreateTablet for tablet cd88d6ef628e4c72960287efda33f710 (DEFAULT_TABLE table=test-table [id=f9a822872c0545b2bf096f8dafa7b17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:27.232146 21856 tablet_service.cc:1467] Processing CreateTablet for tablet cd88d6ef628e4c72960287efda33f710 (DEFAULT_TABLE table=test-table [id=f9a822872c0545b2bf096f8dafa7b17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:27.232416 22002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd88d6ef628e4c72960287efda33f710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:27.232350 21712 tablet_service.cc:1467] Processing CreateTablet for tablet cd88d6ef628e4c72960287efda33f710 (DEFAULT_TABLE table=test-table [id=f9a822872c0545b2bf096f8dafa7b17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:27.232517 21856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd88d6ef628e4c72960287efda33f710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:27.232651 21712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd88d6ef628e4c72960287efda33f710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:27.235116 22094 tablet_bootstrap.cc:492] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6: Bootstrap starting.
I20260810 06:45:27.235872 22095 tablet_bootstrap.cc:492] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff: Bootstrap starting.
I20260810 06:45:27.235991 22094 tablet_bootstrap.cc:654] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:27.236408 22094 log.cc:824] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:27.236603 22095 tablet_bootstrap.cc:654] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:27.236927 22095 log.cc:824] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:27.237323 22096 tablet_bootstrap.cc:492] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901: Bootstrap starting.
I20260810 06:45:27.237497 22094 tablet_bootstrap.cc:492] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6: No bootstrap required, opened a new log
I20260810 06:45:27.237591 22094 ts_tablet_manager.cc:1397] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 06:45:27.237758 22095 tablet_bootstrap.cc:492] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff: No bootstrap required, opened a new log
I20260810 06:45:27.237831 22095 ts_tablet_manager.cc:1397] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:45:27.238034 22096 tablet_bootstrap.cc:654] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:27.238350 22096 log.cc:824] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:27.239146 22096 tablet_bootstrap.cc:492] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901: No bootstrap required, opened a new log
I20260810 06:45:27.239215 22096 ts_tablet_manager.cc:1397] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:45:27.239984 22094 raft_consensus.cc:348] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3625350e8de4831a9026529b24dc901" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36637 } } peers { permanent_uuid: "c75fb251ce5c4179bf9603b6f39323a6" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33681 } } peers { permanent_uuid: "1daaeb50df6a4644b213ce27fafbcdff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 37029 } }
I20260810 06:45:27.240144 22094 raft_consensus.cc:374] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:27.240188 22094 raft_consensus.cc:729] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75fb251ce5c4179bf9603b6f39323a6, State: Initialized, Role: FOLLOWER
I20260810 06:45:27.240339 22094 consensus_queue.cc:260] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3625350e8de4831a9026529b24dc901" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36637 } } peers { permanent_uuid: "c75fb251ce5c4179bf9603b6f39323a6" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33681 } } peers { permanent_uuid: "1daaeb50df6a4644b213ce27fafbcdff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 37029 } }
I20260810 06:45:27.240628 22078 heartbeater.cc:503] Master 127.15.232.254:37085 was elected leader, sending a full tablet report...
I20260810 06:45:27.240625 22094 ts_tablet_manager.cc:1428] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:45:27.240744 22095 raft_consensus.cc:348] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3625350e8de4831a9026529b24dc901" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36637 } } peers { permanent_uuid: "c75fb251ce5c4179bf9603b6f39323a6" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33681 } } peers { permanent_uuid: "1daaeb50df6a4644b213ce27fafbcdff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 37029 } }
I20260810 06:45:27.240891 22095 raft_consensus.cc:374] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:27.240938 22095 raft_consensus.cc:729] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1daaeb50df6a4644b213ce27fafbcdff, State: Initialized, Role: FOLLOWER
I20260810 06:45:27.241083 22095 consensus_queue.cc:260] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3625350e8de4831a9026529b24dc901" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36637 } } peers { permanent_uuid: "c75fb251ce5c4179bf9603b6f39323a6" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33681 } } peers { permanent_uuid: "1daaeb50df6a4644b213ce27fafbcdff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 37029 } }
I20260810 06:45:27.241412 21928 heartbeater.cc:503] Master 127.15.232.254:37085 was elected leader, sending a full tablet report...
I20260810 06:45:27.241407 22095 ts_tablet_manager.cc:1428] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 06:45:27.241914 22096 raft_consensus.cc:348] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3625350e8de4831a9026529b24dc901" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36637 } } peers { permanent_uuid: "c75fb251ce5c4179bf9603b6f39323a6" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33681 } } peers { permanent_uuid: "1daaeb50df6a4644b213ce27fafbcdff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 37029 } }
I20260810 06:45:27.242065 22096 raft_consensus.cc:374] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:27.242100 22096 raft_consensus.cc:729] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3625350e8de4831a9026529b24dc901, State: Initialized, Role: FOLLOWER
I20260810 06:45:27.242197 22096 consensus_queue.cc:260] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3625350e8de4831a9026529b24dc901" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36637 } } peers { permanent_uuid: "c75fb251ce5c4179bf9603b6f39323a6" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33681 } } peers { permanent_uuid: "1daaeb50df6a4644b213ce27fafbcdff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 37029 } }
I20260810 06:45:27.242453 22096 ts_tablet_manager.cc:1428] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 06:45:27.242571 21786 heartbeater.cc:503] Master 127.15.232.254:37085 was elected leader, sending a full tablet report...
W20260810 06:45:27.341034 21929 tablet.cc:2369] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:45:27.422232 22102 raft_consensus.cc:482] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:27.422374 22102 raft_consensus.cc:504] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3625350e8de4831a9026529b24dc901" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36637 } } peers { permanent_uuid: "c75fb251ce5c4179bf9603b6f39323a6" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33681 } } peers { permanent_uuid: "1daaeb50df6a4644b213ce27fafbcdff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 37029 } }
I20260810 06:45:27.422760 22102 leader_election.cc:288] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75fb251ce5c4179bf9603b6f39323a6 (127.15.232.195:33681), 1daaeb50df6a4644b213ce27fafbcdff (127.15.232.194:37029)
I20260810 06:45:27.424255 21788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.423026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:33681 (local address 127.15.232.193:49245)
0810 06:45:27.423098 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:27.423111 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:27.423161 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:27.423356 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:27.423358 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:27.423360 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:27.423853 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:27.423854 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:27.423862 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:27.423877 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:27.424080 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:27.424081 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:27.424082 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 06:45:27.424157 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:45:27.424173 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 06:45:27.424302 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.423078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:49245 (local address 127.15.232.195:33681)
0810 06:45:27.423136 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.423138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.423152 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:45:27.423276 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.423277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.423283 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.423352 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:27.423970 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:27.423972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:27.423972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:27.423976 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:27.423997 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:27.424002 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:27.424067 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:27.424174 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 06:45:27.424196 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:45:27.424420 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.423094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:50275 (local address 127.15.232.194:37029)
0810 06:45:27.423146 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.423148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.423435 (+ 287us) server_negotiation.cc:407] Connection header received
0810 06:45:27.423513 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.423514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.423520 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.423573 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:27.424137 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:27.424139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:27.424140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:27.424145 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:27.424164 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:27.424170 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:27.424226 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:27.424319 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 06:45:27.424334 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:45:27.424609 22103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.423028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:37029 (local address 127.15.232.193:50275)
0810 06:45:27.423345 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:27.423355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:27.423451 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:27.423574 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:27.423575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:27.423577 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:27.424033 (+ 456us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:27.424034 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:27.424040 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:27.424054 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:27.424236 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:27.424237 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:27.424237 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 06:45:27.424521 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 06:45:27.424539 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":136,"thread_start_us":130,"threads_started":1}
I20260810 06:45:27.424609 22025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd88d6ef628e4c72960287efda33f710" candidate_uuid: "a3625350e8de4831a9026529b24dc901" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75fb251ce5c4179bf9603b6f39323a6" is_pre_election: true
I20260810 06:45:27.424762 22025 raft_consensus.cc:2454] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3625350e8de4831a9026529b24dc901 in term 0.
I20260810 06:45:27.424973 21880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd88d6ef628e4c72960287efda33f710" candidate_uuid: "a3625350e8de4831a9026529b24dc901" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1daaeb50df6a4644b213ce27fafbcdff" is_pre_election: true
I20260810 06:45:27.425083 21880 raft_consensus.cc:2454] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3625350e8de4831a9026529b24dc901 in term 0.
I20260810 06:45:27.425031 21660 leader_election.cc:302] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3625350e8de4831a9026529b24dc901, c75fb251ce5c4179bf9603b6f39323a6; no voters:
I20260810 06:45:27.425185 22102 raft_consensus.cc:2781] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:27.425261 22102 raft_consensus.cc:482] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:27.425302 22102 raft_consensus.cc:3037] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:27.426044 22102 raft_consensus.cc:504] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3625350e8de4831a9026529b24dc901" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36637 } } peers { permanent_uuid: "c75fb251ce5c4179bf9603b6f39323a6" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33681 } } peers { permanent_uuid: "1daaeb50df6a4644b213ce27fafbcdff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 37029 } }
I20260810 06:45:27.426256 22102 leader_election.cc:288] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [CANDIDATE]: Term 1 election: Requested vote from peers c75fb251ce5c4179bf9603b6f39323a6 (127.15.232.195:33681), 1daaeb50df6a4644b213ce27fafbcdff (127.15.232.194:37029)
I20260810 06:45:27.426407 22025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd88d6ef628e4c72960287efda33f710" candidate_uuid: "a3625350e8de4831a9026529b24dc901" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75fb251ce5c4179bf9603b6f39323a6"
I20260810 06:45:27.426425 21880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd88d6ef628e4c72960287efda33f710" candidate_uuid: "a3625350e8de4831a9026529b24dc901" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1daaeb50df6a4644b213ce27fafbcdff"
I20260810 06:45:27.426509 22025 raft_consensus.cc:3037] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:27.426512 21880 raft_consensus.cc:3037] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:27.427237 22025 raft_consensus.cc:2454] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3625350e8de4831a9026529b24dc901 in term 1.
I20260810 06:45:27.427320 21880 raft_consensus.cc:2454] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3625350e8de4831a9026529b24dc901 in term 1.
I20260810 06:45:27.427456 21660 leader_election.cc:302] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3625350e8de4831a9026529b24dc901, c75fb251ce5c4179bf9603b6f39323a6; no voters:
I20260810 06:45:27.427556 22102 raft_consensus.cc:2781] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:27.427721 22102 raft_consensus.cc:686] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [term 1 LEADER]: Becoming Leader. State: Replica: a3625350e8de4831a9026529b24dc901, State: Running, Role: LEADER
I20260810 06:45:27.427829 22102 consensus_queue.cc:237] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3625350e8de4831a9026529b24dc901" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36637 } } peers { permanent_uuid: "c75fb251ce5c4179bf9603b6f39323a6" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33681 } } peers { permanent_uuid: "1daaeb50df6a4644b213ce27fafbcdff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 37029 } }
I20260810 06:45:27.428821 21573 catalog_manager.cc:5495] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3625350e8de4831a9026529b24dc901 (127.15.232.193). New cstate: current_term: 1 leader_uuid: "a3625350e8de4831a9026529b24dc901" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3625350e8de4831a9026529b24dc901" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75fb251ce5c4179bf9603b6f39323a6" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1daaeb50df6a4644b213ce27fafbcdff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 37029 } health_report { overall_health: UNKNOWN } } }
I20260810 06:45:27.461144 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.459945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35656 (local address 127.15.232.193:36637)
0810 06:45:27.460015 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.460017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.460029 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:45:27.460138 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.460140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.460158 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.460272 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:27.460888 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:27.460889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:27.460890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:27.460895 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:27.460912 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:27.460917 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:27.460983 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:27.461043 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 06:45:27.461059 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:45:27.463184 22025 raft_consensus.cc:1261] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6 [term 1 FOLLOWER]: Refusing update from remote peer a3625350e8de4831a9026529b24dc901: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:27.463184 21880 raft_consensus.cc:1261] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff [term 1 FOLLOWER]: Refusing update from remote peer a3625350e8de4831a9026529b24dc901: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:27.463593 22104 consensus_queue.cc:1035] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1daaeb50df6a4644b213ce27fafbcdff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:27.463795 22102 consensus_queue.cc:1035] T cd88d6ef628e4c72960287efda33f710 P a3625350e8de4831a9026529b24dc901 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75fb251ce5c4179bf9603b6f39323a6" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 33681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:45:27.463941 22079 tablet.cc:2369] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:45:27.466176 22109 mvcc.cc:205] Tried to move back new op lower bound from 7316866365285769216 to 7316866365144928256. Current Snapshot: MvccSnapshot[applied={T|T < 7316866365285769216}]
I20260810 06:45:27.467325 22110 mvcc.cc:205] Tried to move back new op lower bound from 7316866365285769216 to 7316866365144928256. Current Snapshot: MvccSnapshot[applied={T|T < 7316866365285769216 or (T in {7316866365285769216})}]
I20260810 06:45:27.468770 21573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:27.468876 21573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:27.469705 21573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3b8062ed60e4c9ab409c97030c785ae: Sending DeleteTablet for 3 replicas of tablet cd88d6ef628e4c72960287efda33f710
I20260810 06:45:27.470043 22002 tablet_service.cc:1514] Processing DeleteTablet for tablet cd88d6ef628e4c72960287efda33f710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:27 UTC) from {username='slave'} at 127.0.0.1:49660
I20260810 06:45:27.470068 21712 tablet_service.cc:1514] Processing DeleteTablet for tablet cd88d6ef628e4c72960287efda33f710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:27 UTC) from {username='slave'} at 127.0.0.1:35644
I20260810 06:45:27.470158 21856 tablet_service.cc:1514] Processing DeleteTablet for tablet cd88d6ef628e4c72960287efda33f710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:27 UTC) from {username='slave'} at 127.0.0.1:34100
I20260810 06:45:27.470299 22120 tablet_replica.cc:330] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6: stopping tablet replica
I20260810 06:45:27.470404 22120 raft_consensus.cc:2229] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:27.470505 22120 raft_consensus.cc:2258] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:27.470587 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 21644
I20260810 06:45:27.470661 22121 tablet_replica.cc:330] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff: stopping tablet replica
I20260810 06:45:27.470808 22121 raft_consensus.cc:2229] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:27.470901 22121 raft_consensus.cc:2258] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:27.471527 22121 ts_tablet_manager.cc:1905] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:27.472361 22120 ts_tablet_manager.cc:1905] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:27.473853 22121 ts_tablet_manager.cc:1918] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:27.474004 22121 log.cc:1196] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/cd88d6ef628e4c72960287efda33f710
I20260810 06:45:27.474002 22120 ts_tablet_manager.cc:1918] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:27.474072 22120 log.cc:1196] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/cd88d6ef628e4c72960287efda33f710
I20260810 06:45:27.474360 22121 ts_tablet_manager.cc:1939] T cd88d6ef628e4c72960287efda33f710 P 1daaeb50df6a4644b213ce27fafbcdff: Deleting consensus metadata
I20260810 06:45:27.474361 22120 ts_tablet_manager.cc:1939] T cd88d6ef628e4c72960287efda33f710 P c75fb251ce5c4179bf9603b6f39323a6: Deleting consensus metadata
I20260810 06:45:27.474804 21556 catalog_manager.cc:4841] TS c75fb251ce5c4179bf9603b6f39323a6 (127.15.232.195:33681): tablet cd88d6ef628e4c72960287efda33f710 (table test-table [id=f9a822872c0545b2bf096f8dafa7b17f]) successfully deleted
I20260810 06:45:27.474960 21557 catalog_manager.cc:4841] TS 1daaeb50df6a4644b213ce27fafbcdff (127.15.232.194:37029): tablet cd88d6ef628e4c72960287efda33f710 (table test-table [id=f9a822872c0545b2bf096f8dafa7b17f]) successfully deleted
W20260810 06:45:27.483599 21557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:36637 (error 108)
I20260810 06:45:27.484145 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 21789
I20260810 06:45:27.484206 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.484063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:36637 (local address 127.0.0.1:35668)
0810 06:45:27.484137 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:27.484178 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:36637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:45:27.486073 21557 catalog_manager.cc:4584] TS a3625350e8de4831a9026529b24dc901 (127.15.232.193:36637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cd88d6ef628e4c72960287efda33f710: Network error: Client connection negotiation failed: client connection to 127.15.232.193:36637: connect: Connection refused (error 111)
I20260810 06:45:27.493739 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 21932
I20260810 06:45:27.499331 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 21540
2026-08-10T06:45:27Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (817 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 06:45:27.522030 16291 mini_oidc.cc:152] Starting JWKS server
I20260810 06:45:27.524695 16291 webserver.cc:460] Webserver started at https://127.0.0.1:40127/ using document root <none> and password file <none>
I20260810 06:45:27.524832 16291 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:45:27.525036 16291 webserver.cc:460] Webserver started at http://127.0.0.1:39259/ using document root <none> and password file <none>
2026-08-10T06:45:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:27Z Disabled control of system clock
I20260810 06:45:27.533645 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:44571
--webserver_interface=127.15.232.254
--webserver_port=0
--builtin_ntp_servers=127.15.232.212:43099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:44571
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40127/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/testchainca.cert with env {}
W20260810 06:45:27.615038 22128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:27.615262 22128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:27.615286 22128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:27.616428 22128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:27.616464 22128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:27.616477 22128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:27.616489 22128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:27.616501 22128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:45:27.616513 22128 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:45:27.616524 22128 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40127/jwks.json
I20260810 06:45:27.618145 22128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:43099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:44571
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:44571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40127/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:27.618350 22128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:27.618518 22128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:27.623364 22135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:27.623385 22133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:27.623371 22137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:27.623577 22128 server_base.cc:1034] running on GCE node
I20260810 06:45:27.623956 22128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:27.624238 22128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:27.625411 22128 hybrid_clock.cc:648] HybridClock initialized: now 1786344327625364 us; error 76 us; skew 500 ppm
I20260810 06:45:27.626703 22128 webserver.cc:460] Webserver started at http://127.15.232.254:33537/ using document root <none> and password file <none>
I20260810 06:45:27.626988 22128 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:27.627059 22128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:27.627213 22128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:27.628343 22128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "7d183054e0194367b709d983eca2d0c1"
format_stamp: "Formatted at 2026-08-10 06:45:27 on dist-test-slave-vxj2"
server_key: "248751f38479d36d86c8df998564b511"
server_key_iv: "4c6a742024b0b0e20fb3064ebdeab7cf"
server_key_version: "encryptionkey@0"
I20260810 06:45:27.628728 22128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "7d183054e0194367b709d983eca2d0c1"
format_stamp: "Formatted at 2026-08-10 06:45:27 on dist-test-slave-vxj2"
server_key: "248751f38479d36d86c8df998564b511"
server_key_iv: "4c6a742024b0b0e20fb3064ebdeab7cf"
server_key_version: "encryptionkey@0"
I20260810 06:45:27.630576 22128 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260810 06:45:27.631532 22146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:27.631762 22128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:27.631862 22128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "7d183054e0194367b709d983eca2d0c1"
format_stamp: "Formatted at 2026-08-10 06:45:27 on dist-test-slave-vxj2"
server_key: "248751f38479d36d86c8df998564b511"
server_key_iv: "4c6a742024b0b0e20fb3064ebdeab7cf"
server_key_version: "encryptionkey@0"
I20260810 06:45:27.631937 22128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:27.645319 22128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:27.645643 22128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:27.645794 22128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:27.649987 22128 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:44571
I20260810 06:45:27.650044 22208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:44571 every 8 connection(s)
I20260810 06:45:27.650405 22128 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:27.651448 22209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:27.654068 22209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1: Bootstrap starting.
I20260810 06:45:27.654718 22209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:27.655063 22209 log.cc:824] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:27.655855 22209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1: No bootstrap required, opened a new log
I20260810 06:45:27.657650 22209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d183054e0194367b709d983eca2d0c1" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44571 } }
I20260810 06:45:27.657814 22209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:27.657840 22209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d183054e0194367b709d983eca2d0c1, State: Initialized, Role: FOLLOWER
I20260810 06:45:27.658056 22209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d183054e0194367b709d983eca2d0c1" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44571 } }
I20260810 06:45:27.658181 22209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:27.658224 22209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:27.658248 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 22128
I20260810 06:45:27.658283 22209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:27.658355 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:27.658556 16291 external_mini_cluster.cc:1428] Setting key 0ead7bd9ae53f947ace2f5b3af4e9f3b
I20260810 06:45:27.659000 22209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d183054e0194367b709d983eca2d0c1" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44571 } }
I20260810 06:45:27.659123 22209 leader_election.cc:302] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d183054e0194367b709d983eca2d0c1; no voters:
I20260810 06:45:27.659291 22209 leader_election.cc:288] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:27.659320 22214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:27.659466 22214 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [term 1 LEADER]: Becoming Leader. State: Replica: 7d183054e0194367b709d983eca2d0c1, State: Running, Role: LEADER
I20260810 06:45:27.659556 22214 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d183054e0194367b709d983eca2d0c1" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44571 } }
I20260810 06:45:27.659606 22209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:27.659865 22214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d183054e0194367b709d983eca2d0c1. Latest consensus state: current_term: 1 leader_uuid: "7d183054e0194367b709d983eca2d0c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d183054e0194367b709d983eca2d0c1" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44571 } } }
I20260810 06:45:27.659955 22214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:27.660149 22218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:27.660655 22218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:27.660091 22215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d183054e0194367b709d983eca2d0c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d183054e0194367b709d983eca2d0c1" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44571 } } }
I20260810 06:45:27.661464 22215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:27.662335 22232 mvcc.cc:205] Tried to move back new op lower bound from 7316866366103146496 to 7316866366093770752. Current Snapshot: MvccSnapshot[applied={T|T < 7316866366103146496}]
I20260810 06:45:27.662743 22218 catalog_manager.cc:1347] Generated new cluster ID: 06d56b68b3ed4983ab5ddeb2ba792fd4
I20260810 06:45:27.662796 22218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:27.664988 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.659082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58332 (local address 127.15.232.254:44571)
0810 06:45:27.659306 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.659315 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.659356 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:45:27.659479 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.659487 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.659572 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.659997 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:27.660107 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:27.661520 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:27.662671 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:27.663249 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:27.663373 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:27.664178 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:27.664182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:27.664182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:27.664388 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:27.664428 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:27.664442 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:27.664504 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:27.664623 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 06:45:27.664784 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260810 06:45:27.665913 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:45:27.673086 22218 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:27.673923 22218 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:27.686003 22218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d183054e0194367b709d983eca2d0c1: Generated new TSK 0
I20260810 06:45:27.686261 22218 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 06:45:27.719528 22243 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 06:45:27.719698 22243 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 06:45:27.719941 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.696226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.15.232.254:44571)
0810 06:45:27.696300 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.696318 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.696451 (+ 133us) server_negotiation.cc:407] Connection header received
0810 06:45:27.696604 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.696605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.696623 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.696695 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:45:27.696900 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:27.697458 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:27.698272 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:27.698464 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:27.698539 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:27.698781 (+ 242us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:45:27.719679 (+ 20898us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40127/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 06:45:27.719858 (+ 179us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58346: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40127/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 06:45:27.720008 22213 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58346: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40127/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 06:45:27.720448 22242 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:27.696157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44571 (local address 127.0.0.1:58346)
0810 06:45:27.696378 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:27.696388 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:27.696484 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:27.696707 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:27.696708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:27.696716 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:45:27.696790 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:27.696793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:27.697548 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:27.697549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:27.698168 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:27.698171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:27.698554 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:27.698555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:27.698624 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:27.698644 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:45:27.719912 (+ 21268us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40127/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 06:45:27.720056 (+ 144us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.232.254:44571: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40127/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":106,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.232.254:44571: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40127/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 06:45:27.720975 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 22128
2026-08-10T06:45:27Z chronyd exiting
I20260810 06:45:27.759034 16291 test_util.cc:175] -----------------------------------------------
I20260810 06:45:27.759145 16291 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344303981681-16291-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (238 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 06:45:27.760057 16291 mini_oidc.cc:152] Starting JWKS server
I20260810 06:45:27.760775 16291 webserver.cc:460] Webserver started at https://127.0.0.1:33981/ using document root <none> and password file <none>
I20260810 06:45:27.760849 16291 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:45:27.761087 16291 webserver.cc:460] Webserver started at http://127.0.0.1:34943/ using document root <none> and password file <none>
2026-08-10T06:45:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:27Z Disabled control of system clock
I20260810 06:45:27.769199 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:45817
--webserver_interface=127.15.232.254
--webserver_port=0
--builtin_ntp_servers=127.15.232.212:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:45817
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33981/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/testchainca.cert with env {}
W20260810 06:45:27.850740 22249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:27.850965 22249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:27.850988 22249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:27.852177 22249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:27.852226 22249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:27.852239 22249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:27.852252 22249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:27.852263 22249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:45:27.852275 22249 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:45:27.852288 22249 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33981/jwks.json
I20260810 06:45:27.853942 22249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:45817
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:45817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33981/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:27.854202 22249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:27.854421 22249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:27.859468 22256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:27.859579 22255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:27.859484 22258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:27.859743 22249 server_base.cc:1034] running on GCE node
I20260810 06:45:27.860028 22249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:27.860268 22249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:27.861409 22249 hybrid_clock.cc:648] HybridClock initialized: now 1786344327861387 us; error 48 us; skew 500 ppm
I20260810 06:45:27.862617 22249 webserver.cc:460] Webserver started at http://127.15.232.254:44731/ using document root <none> and password file <none>
I20260810 06:45:27.862807 22249 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:27.862890 22249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:27.862982 22249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:27.864012 22249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "071f1bc967a2481a8c698e5e5b4d85a6"
format_stamp: "Formatted at 2026-08-10 06:45:27 on dist-test-slave-vxj2"
server_key: "4ddf4ea1edad6d2fa78a9a5887a064c4"
server_key_iv: "7829b3df5a7677c012205166ea5133d7"
server_key_version: "encryptionkey@0"
I20260810 06:45:27.864313 22249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "071f1bc967a2481a8c698e5e5b4d85a6"
format_stamp: "Formatted at 2026-08-10 06:45:27 on dist-test-slave-vxj2"
server_key: "4ddf4ea1edad6d2fa78a9a5887a064c4"
server_key_iv: "7829b3df5a7677c012205166ea5133d7"
server_key_version: "encryptionkey@0"
I20260810 06:45:27.865679 22249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:27.866449 22265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:27.866643 22249 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:27.866732 22249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "071f1bc967a2481a8c698e5e5b4d85a6"
format_stamp: "Formatted at 2026-08-10 06:45:27 on dist-test-slave-vxj2"
server_key: "4ddf4ea1edad6d2fa78a9a5887a064c4"
server_key_iv: "7829b3df5a7677c012205166ea5133d7"
server_key_version: "encryptionkey@0"
I20260810 06:45:27.866788 22249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:27.882138 22249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:27.882416 22249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:27.882521 22249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:27.886289 22249 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:45817
I20260810 06:45:27.886363 22324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:45817 every 8 connection(s)
I20260810 06:45:27.886714 22249 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:27.887409 22325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:27.890522 22325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6: Bootstrap starting.
I20260810 06:45:27.891125 22325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:27.891413 22325 log.cc:824] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:27.892102 22325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6: No bootstrap required, opened a new log
I20260810 06:45:27.893690 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 22249
I20260810 06:45:27.893783 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:27.894008 16291 external_mini_cluster.cc:1428] Setting key 67f5648bc78747058da0b072ad8a4eee
I20260810 06:45:27.893940 22325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071f1bc967a2481a8c698e5e5b4d85a6" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 45817 } }
I20260810 06:45:27.894106 22325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:27.894155 22325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 071f1bc967a2481a8c698e5e5b4d85a6, State: Initialized, Role: FOLLOWER
I20260810 06:45:27.894294 22325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071f1bc967a2481a8c698e5e5b4d85a6" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 45817 } }
I20260810 06:45:27.894385 22325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:27.894423 22325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:27.894470 22325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:27.895062 22325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071f1bc967a2481a8c698e5e5b4d85a6" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 45817 } }
I20260810 06:45:27.895190 22325 leader_election.cc:302] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 071f1bc967a2481a8c698e5e5b4d85a6; no voters:
I20260810 06:45:27.895386 22325 leader_election.cc:288] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:27.895454 22330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:27.895615 22330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [term 1 LEADER]: Becoming Leader. State: Replica: 071f1bc967a2481a8c698e5e5b4d85a6, State: Running, Role: LEADER
I20260810 06:45:27.895656 22325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:27.895702 22330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071f1bc967a2481a8c698e5e5b4d85a6" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 45817 } }
I20260810 06:45:27.896065 22330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 071f1bc967a2481a8c698e5e5b4d85a6. Latest consensus state: current_term: 1 leader_uuid: "071f1bc967a2481a8c698e5e5b4d85a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071f1bc967a2481a8c698e5e5b4d85a6" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 45817 } } }
I20260810 06:45:27.896082 22332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "071f1bc967a2481a8c698e5e5b4d85a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071f1bc967a2481a8c698e5e5b4d85a6" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 45817 } } }
I20260810 06:45:27.896164 22330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:27.896195 22332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:27.896605 22337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:27.897081 22337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:27.898591 22337 catalog_manager.cc:1347] Generated new cluster ID: dbca8a1bb6d745d4aab38781b24592e3
I20260810 06:45:27.898648 22337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:27.900822 22329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.894710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54370 (local address 127.15.232.254:45817)
0810 06:45:27.894943 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.894950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.894973 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:45:27.895064 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.895069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.895124 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.895264 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:27.895581 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:27.897545 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:27.898646 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:27.899224 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:27.899341 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:27.900081 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:27.900084 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:27.900085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:27.900301 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:27.900340 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:27.900354 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:27.900423 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:27.900531 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 06:45:27.900613 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260810 06:45:27.901746 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:45:27.931770 22337 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:27.932556 22337 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:27.943378 22337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 071f1bc967a2481a8c698e5e5b4d85a6: Generated new TSK 0
I20260810 06:45:27.943645 22337 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 06:45:27.970593 22359 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 06:45:27.970695 22359 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 06:45:27.971022 22329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:27.949629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.15.232.254:45817)
0810 06:45:27.949689 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:27.949692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:27.949834 (+ 142us) server_negotiation.cc:407] Connection header received
0810 06:45:27.949912 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:27.949913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:27.949925 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:27.949986 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:45:27.950144 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:27.950641 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:27.951806 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:27.952005 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:27.952059 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:27.952257 (+ 198us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:45:27.970748 (+ 18491us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33981/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 06:45:27.970949 (+ 201us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54372: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33981/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":34}
W20260810 06:45:27.971092 22329 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54372: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33981/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 06:45:27.971102 22358 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:27.949531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:45817 (local address 127.0.0.1:54372)
0810 06:45:27.949752 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:27.949764 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:27.949836 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:27.950007 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:27.950008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:27.950010 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:45:27.950077 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:27.950080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:27.950734 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:27.950735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:27.951714 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:27.951717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:27.952084 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:27.952085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:27.952148 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:27.952154 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:45:27.970866 (+ 18712us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33981/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 06:45:27.971004 (+ 138us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.232.254:45817: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33981/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":110,"threads_started":1}
I20260810 06:45:27.971391 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 22249
2026-08-10T06:45:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (255 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 06:45:28.016068 16291 mini_oidc.cc:152] Starting JWKS server
I20260810 06:45:28.016822 16291 webserver.cc:460] Webserver started at https://127.0.0.1:33311/ using document root <none> and password file <none>
I20260810 06:45:28.016868 16291 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:45:28.017074 16291 webserver.cc:460] Webserver started at http://127.0.0.1:34293/ using document root <none> and password file <none>
2026-08-10T06:45:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:28Z Disabled control of system clock
I20260810 06:45:28.025776 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:39697
--webserver_interface=127.15.232.254
--webserver_port=0
--builtin_ntp_servers=127.15.232.212:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:39697
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33311/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 06:45:28.108439 22365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:28.108664 22365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:28.108685 22365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:28.109927 22365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:28.109975 22365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:28.109988 22365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:28.110001 22365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:28.110047 22365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:45:28.110080 22365 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:45:28.110096 22365 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33311/jwks.json
I20260810 06:45:28.111507 22365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:39697
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:39697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33311/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:28.111802 22365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:28.112116 22365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:28.116997 22373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:28.117093 22371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:28.117007 22370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:28.117336 22365 server_base.cc:1034] running on GCE node
I20260810 06:45:28.117627 22365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:28.117986 22365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:28.119133 22365 hybrid_clock.cc:648] HybridClock initialized: now 1786344328119123 us; error 30 us; skew 500 ppm
I20260810 06:45:28.120513 22365 webserver.cc:460] Webserver started at http://127.15.232.254:34551/ using document root <none> and password file <none>
I20260810 06:45:28.120783 22365 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:28.120833 22365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:28.121009 22365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:28.122156 22365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "ed982bd93f7049ffaafdf0bfc3ec25d2"
format_stamp: "Formatted at 2026-08-10 06:45:28 on dist-test-slave-vxj2"
server_key: "0bd4c19ca12de9edbecf1093dad8772a"
server_key_iv: "e744f062b4300ead030b8f3bea97892a"
server_key_version: "encryptionkey@0"
I20260810 06:45:28.122529 22365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "ed982bd93f7049ffaafdf0bfc3ec25d2"
format_stamp: "Formatted at 2026-08-10 06:45:28 on dist-test-slave-vxj2"
server_key: "0bd4c19ca12de9edbecf1093dad8772a"
server_key_iv: "e744f062b4300ead030b8f3bea97892a"
server_key_version: "encryptionkey@0"
I20260810 06:45:28.123879 22365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:28.124615 22379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:28.124831 22365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:28.124939 22365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "ed982bd93f7049ffaafdf0bfc3ec25d2"
format_stamp: "Formatted at 2026-08-10 06:45:28 on dist-test-slave-vxj2"
server_key: "0bd4c19ca12de9edbecf1093dad8772a"
server_key_iv: "e744f062b4300ead030b8f3bea97892a"
server_key_version: "encryptionkey@0"
I20260810 06:45:28.125026 22365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:28.143116 22365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:28.143488 22365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:28.143659 22365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:28.148056 22365 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:39697
I20260810 06:45:28.148097 22435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:39697 every 8 connection(s)
I20260810 06:45:28.148463 22365 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:28.149253 22436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:28.150297 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 22365
I20260810 06:45:28.150419 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:28.150616 16291 external_mini_cluster.cc:1428] Setting key 21feebb68b07c3c794e53ab9f0f25d00
I20260810 06:45:28.152715 22436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2: Bootstrap starting.
I20260810 06:45:28.153441 22436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:28.153762 22436 log.cc:824] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:28.154564 22436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2: No bootstrap required, opened a new log
I20260810 06:45:28.156394 22440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:28.151249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56878 (local address 127.15.232.254:39697)
0810 06:45:28.151469 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:28.151477 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:28.151502 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:45:28.151915 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:28.151922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:28.151983 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:28.152099 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:28.152378 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:28.153346 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:28.154360 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:28.154864 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:28.154971 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:28.155687 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:28.155691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:28.155691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:28.155864 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:28.155899 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:28.155910 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:28.155970 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:28.156084 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 06:45:28.156173 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260810 06:45:28.156899 22436 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed982bd93f7049ffaafdf0bfc3ec25d2" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39697 } }
I20260810 06:45:28.157087 22436 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:28.157114 22436 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed982bd93f7049ffaafdf0bfc3ec25d2, State: Initialized, Role: FOLLOWER
I20260810 06:45:28.157306 22436 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed982bd93f7049ffaafdf0bfc3ec25d2" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39697 } }
I20260810 06:45:28.157405 22436 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:28.157440 22436 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:28.157495 22436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:28.157577 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:45:28.158164 22436 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed982bd93f7049ffaafdf0bfc3ec25d2" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39697 } }
I20260810 06:45:28.158294 22436 leader_election.cc:302] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed982bd93f7049ffaafdf0bfc3ec25d2; no voters:
I20260810 06:45:28.158450 22436 leader_election.cc:288] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:28.158492 22442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:28.158697 22442 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [term 1 LEADER]: Becoming Leader. State: Replica: ed982bd93f7049ffaafdf0bfc3ec25d2, State: Running, Role: LEADER
I20260810 06:45:28.158746 22436 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:28.158797 22442 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed982bd93f7049ffaafdf0bfc3ec25d2" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39697 } }
I20260810 06:45:28.159242 22444 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed982bd93f7049ffaafdf0bfc3ec25d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed982bd93f7049ffaafdf0bfc3ec25d2" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39697 } } }
I20260810 06:45:28.159253 22443 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed982bd93f7049ffaafdf0bfc3ec25d2. Latest consensus state: current_term: 1 leader_uuid: "ed982bd93f7049ffaafdf0bfc3ec25d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed982bd93f7049ffaafdf0bfc3ec25d2" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39697 } } }
I20260810 06:45:28.159335 22444 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:28.159389 22443 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:28.159583 22451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:28.160271 22451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:28.161926 22451 catalog_manager.cc:1347] Generated new cluster ID: 84d689e6a79c44db836a3e6391dcc366
I20260810 06:45:28.161981 22451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:28.178792 22451 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:28.179529 22451 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:28.183555 22451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed982bd93f7049ffaafdf0bfc3ec25d2: Generated new TSK 0
I20260810 06:45:28.183740 22451 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:45:28.204469 22440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:28.197519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56888 (local address 127.15.232.254:39697)
0810 06:45:28.197576 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:28.197579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:28.199559 (+ 1980us) server_negotiation.cc:407] Connection header received
0810 06:45:28.199729 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:28.199731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:28.199745 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:28.199798 (+ 53us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:45:28.200105 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:28.200976 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:28.202231 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:28.202605 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:28.202718 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:28.202950 (+ 232us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:45:28.204198 (+ 1248us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33311/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 06:45:28.204367 (+ 169us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33311/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":28}
W20260810 06:45:28.204552 22440 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33311/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 06:45:28.209602 22470 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:28.197437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:39697 (local address 127.0.0.1:56888)
0810 06:45:28.199442 (+ 2005us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:28.199456 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:28.199598 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:28.199909 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:28.199910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:28.199912 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:45:28.199986 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:28.199990 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:28.201100 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:28.201101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:28.202115 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:28.202119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:28.202720 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:28.202721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:28.202790 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:28.202797 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:45:28.209292 (+ 6495us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33311/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 06:45:28.209456 (+ 164us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.232.254:39697: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33311/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":1969,"thread_start_us":135,"threads_started":1}
I20260810 06:45:28.210115 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 22365
E20260810 06:45:28.220213 22471 webserver.cc:533] Webserver: sslize failed:
E20260810 06:45:28.220324 22471 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T06:45:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (236 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T06:45:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:28Z Disabled control of system clock
I20260810 06:45:28.263860 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:37263
--webserver_interface=127.15.232.254
--webserver_port=0
--builtin_ntp_servers=127.15.232.212:39593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:37263
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/testchainca.cert with env {}
W20260810 06:45:28.359364 22475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:28.359586 22475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:28.359609 22475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:28.360898 22475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:28.360950 22475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:28.360963 22475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:28.360976 22475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:28.360988 22475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:45:28.361001 22475 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:45:28.361012 22475 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 06:45:28.362682 22475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:39593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:37263
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:37263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:28.362973 22475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:28.363204 22475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:28.368734 22480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:28.368801 22483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:28.368827 22481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:28.368851 22475 server_base.cc:1034] running on GCE node
I20260810 06:45:28.369320 22475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:28.369619 22475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:28.370815 22475 hybrid_clock.cc:648] HybridClock initialized: now 1786344328370780 us; error 49 us; skew 500 ppm
I20260810 06:45:28.372189 22475 webserver.cc:460] Webserver started at http://127.15.232.254:36913/ using document root <none> and password file <none>
I20260810 06:45:28.372500 22475 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:28.372576 22475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:28.372733 22475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:28.374377 22475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "34ac3dd931c84ddd98861f570e294c0d"
format_stamp: "Formatted at 2026-08-10 06:45:28 on dist-test-slave-vxj2"
server_key: "0bf62ee8f73148e5255810cbb0fc4001"
server_key_iv: "111b5d632068b8cbafe982543ae3fd69"
server_key_version: "encryptionkey@0"
I20260810 06:45:28.374859 22475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "34ac3dd931c84ddd98861f570e294c0d"
format_stamp: "Formatted at 2026-08-10 06:45:28 on dist-test-slave-vxj2"
server_key: "0bf62ee8f73148e5255810cbb0fc4001"
server_key_iv: "111b5d632068b8cbafe982543ae3fd69"
server_key_version: "encryptionkey@0"
I20260810 06:45:28.376350 22475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:28.377357 22490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:28.377552 22475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:28.377671 22475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "34ac3dd931c84ddd98861f570e294c0d"
format_stamp: "Formatted at 2026-08-10 06:45:28 on dist-test-slave-vxj2"
server_key: "0bf62ee8f73148e5255810cbb0fc4001"
server_key_iv: "111b5d632068b8cbafe982543ae3fd69"
server_key_version: "encryptionkey@0"
I20260810 06:45:28.377808 22475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:28.406011 22475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:28.406394 22475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:28.406594 22475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:28.411047 22475 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:37263
I20260810 06:45:28.411085 22553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:37263 every 8 connection(s)
I20260810 06:45:28.411463 22475 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:28.412518 22554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:28.415123 22554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d: Bootstrap starting.
I20260810 06:45:28.415338 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 22475
I20260810 06:45:28.415453 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:28.415639 16291 external_mini_cluster.cc:1428] Setting key 21dc04c2dd1b62cf0f723ae19ad66a2b
I20260810 06:45:28.415931 22554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:28.416365 22554 log.cc:824] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:28.417290 22554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d: No bootstrap required, opened a new log
I20260810 06:45:28.419857 22554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ac3dd931c84ddd98861f570e294c0d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37263 } }
I20260810 06:45:28.420073 22554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:28.420114 22554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34ac3dd931c84ddd98861f570e294c0d, State: Initialized, Role: FOLLOWER
I20260810 06:45:28.420246 22554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ac3dd931c84ddd98861f570e294c0d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37263 } }
I20260810 06:45:28.420352 22554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:28.420387 22554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:28.420436 22554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:28.421216 22554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ac3dd931c84ddd98861f570e294c0d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37263 } }
I20260810 06:45:28.421365 22554 leader_election.cc:302] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34ac3dd931c84ddd98861f570e294c0d; no voters:
I20260810 06:45:28.421578 22554 leader_election.cc:288] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:28.421612 22559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:28.421794 22559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [term 1 LEADER]: Becoming Leader. State: Replica: 34ac3dd931c84ddd98861f570e294c0d, State: Running, Role: LEADER
I20260810 06:45:28.421880 22554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:28.421885 22559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ac3dd931c84ddd98861f570e294c0d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37263 } }
I20260810 06:45:28.422477 22561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34ac3dd931c84ddd98861f570e294c0d. Latest consensus state: current_term: 1 leader_uuid: "34ac3dd931c84ddd98861f570e294c0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ac3dd931c84ddd98861f570e294c0d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37263 } } }
I20260810 06:45:28.422573 22561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:28.422484 22560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34ac3dd931c84ddd98861f570e294c0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ac3dd931c84ddd98861f570e294c0d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37263 } } }
I20260810 06:45:28.422668 22560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:28.422521 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:28.416165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41216 (local address 127.15.232.254:37263)
0810 06:45:28.416455 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:28.416465 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:28.416500 (+ 35us) server_negotiation.cc:407] Connection header received
0810 06:45:28.417159 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:28.417195 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:28.417283 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:28.417465 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:28.417651 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:28.419163 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:28.419956 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:28.420669 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:28.420826 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:28.421530 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:28.421534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:28.421535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:28.421786 (+ 251us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:28.421856 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:28.421873 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:28.421951 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:28.422059 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 06:45:28.422282 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":178,"thread_start_us":110,"threads_started":1}
I20260810 06:45:28.423661 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:45:28.423552 22567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:28.424211 22567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:28.425657 22567 catalog_manager.cc:1347] Generated new cluster ID: 17b0345414ed4473bfd78a5481e2e40c
I20260810 06:45:28.425709 22567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:28.439622 22567 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:28.440203 22567 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:28.442744 22567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34ac3dd931c84ddd98861f570e294c0d: Generated new TSK 0
I20260810 06:45:28.442926 22567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:45:28.518702 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:28.450185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41218 (local address 127.15.232.254:37263)
0810 06:45:28.450250 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:28.450253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:28.450386 (+ 133us) server_negotiation.cc:407] Connection header received
0810 06:45:28.450466 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:28.450467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:28.450477 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:28.450537 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:45:28.450700 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:28.451453 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:28.452786 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:28.452989 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:28.453045 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:28.453284 (+ 239us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:45:28.518479 (+ 65195us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 06:45:28.518643 (+ 164us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41218: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 06:45:28.518813 22557 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41218: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 06:45:28.518949 22586 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:28.450125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:37263 (local address 127.0.0.1:41218)
0810 06:45:28.450298 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:28.450311 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:28.450387 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:28.450559 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:28.450561 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:28.450563 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:45:28.450632 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:28.450635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:28.451564 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:28.451565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:28.452692 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:28.452696 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:28.453072 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:28.453074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:28.453147 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:28.453180 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:45:28.518664 (+ 65484us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 06:45:28.518833 (+ 169us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.232.254:37263: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260810 06:45:28.519320 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 22475
2026-08-10T06:45:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (286 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (89 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:28 dist-test-slave-vxj2 krb5kdc[22598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:28 dist-test-slave-vxj2 krb5kdc[22598](info): set up 2 sockets
Aug 10 06:45:28 dist-test-slave-vxj2 krb5kdc[22598](info): commencing operation
krb5kdc: starting...
W20260810 06:45:29.829543 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:29 dist-test-slave-vxj2 krb5kdc[22598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:45:29.957877 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:32891
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:32891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:30.043198 22612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:30.043414 22612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:30.043438 22612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:30.044657 22612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:30.044710 22612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:30.044724 22612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:30.044739 22612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:30.044750 22612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:30.046406 22612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:32891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:32891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:30.046701 22612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:30.046970 22612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:30.052194 22622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:30.052249 22617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:30.052260 22618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:30.052445 22612 server_base.cc:1034] running on GCE node
I20260810 06:45:30.052743 22612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:30.053061 22612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:30.054260 22612 hybrid_clock.cc:648] HybridClock initialized: now 1786344330054253 us; error 69 us; skew 500 ppm
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:30.057040 22612 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:45:30.057521 22612 webserver.cc:460] Webserver started at http://127.15.232.254:46107/ using document root <none> and password file <none>
I20260810 06:45:30.057785 22612 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:30.057855 22612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:30.058015 22612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:30.059103 22612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "6e3eeb8a2da94c589b019531c11c468d"
format_stamp: "Formatted at 2026-08-10 06:45:30 on dist-test-slave-vxj2"
server_key: "f283cb38fe7bd45afe35af88e665ff8f"
server_key_iv: "d9f275268a8fc4eb5f2da43a6c2147dd"
server_key_version: "encryptionkey@0"
I20260810 06:45:30.059489 22612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "6e3eeb8a2da94c589b019531c11c468d"
format_stamp: "Formatted at 2026-08-10 06:45:30 on dist-test-slave-vxj2"
server_key: "f283cb38fe7bd45afe35af88e665ff8f"
server_key_iv: "d9f275268a8fc4eb5f2da43a6c2147dd"
server_key_version: "encryptionkey@0"
I20260810 06:45:30.060967 22612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:30.061934 22631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:30.062155 22612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:30.062273 22612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "6e3eeb8a2da94c589b019531c11c468d"
format_stamp: "Formatted at 2026-08-10 06:45:30 on dist-test-slave-vxj2"
server_key: "f283cb38fe7bd45afe35af88e665ff8f"
server_key_iv: "d9f275268a8fc4eb5f2da43a6c2147dd"
server_key_version: "encryptionkey@0"
I20260810 06:45:30.062356 22612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:30.076186 22612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:30.078827 22612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:30.079012 22612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:30.083700 22612 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:32891
I20260810 06:45:30.083738 22690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:32891 every 8 connection(s)
I20260810 06:45:30.084127 22612 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:30.085124 22691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:30.087599 22691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d: Bootstrap starting.
I20260810 06:45:30.088184 22691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:30.088436 22691 log.cc:824] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:30.089282 22691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d: No bootstrap required, opened a new log
I20260810 06:45:30.090946 22691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3eeb8a2da94c589b019531c11c468d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 32891 } }
I20260810 06:45:30.091095 22691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:30.091117 22691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e3eeb8a2da94c589b019531c11c468d, State: Initialized, Role: FOLLOWER
I20260810 06:45:30.091266 22691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3eeb8a2da94c589b019531c11c468d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 32891 } }
I20260810 06:45:30.091353 22691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:30.091377 22691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:30.091408 22691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:30.091941 22691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3eeb8a2da94c589b019531c11c468d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 32891 } }
I20260810 06:45:30.092031 22691 leader_election.cc:302] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e3eeb8a2da94c589b019531c11c468d; no voters:
I20260810 06:45:30.092178 22691 leader_election.cc:288] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:30.092260 22694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:30.092469 22691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:30.092711 22694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [term 1 LEADER]: Becoming Leader. State: Replica: 6e3eeb8a2da94c589b019531c11c468d, State: Running, Role: LEADER
I20260810 06:45:30.092818 22694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3eeb8a2da94c589b019531c11c468d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 32891 } }
I20260810 06:45:30.093223 22695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e3eeb8a2da94c589b019531c11c468d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3eeb8a2da94c589b019531c11c468d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 32891 } } }
I20260810 06:45:30.093340 22695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:30.093532 22696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e3eeb8a2da94c589b019531c11c468d. Latest consensus state: current_term: 1 leader_uuid: "6e3eeb8a2da94c589b019531c11c468d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3eeb8a2da94c589b019531c11c468d" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 32891 } } }
I20260810 06:45:30.093572 22705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:30.093605 22696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:30.093693 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 22612
I20260810 06:45:30.093815 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:30.094054 16291 external_mini_cluster.cc:1428] Setting key d8a9e112d451fe70d41f85a2cc4fd5a5
I20260810 06:45:30.094211 22705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:30.096074 22705 catalog_manager.cc:1347] Generated new cluster ID: 9f5105e7fa76451cb0ac06a8fb5049f8
I20260810 06:45:30.096132 22705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:30.105018 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:30.094807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44392 (local address 127.15.232.254:32891)
0810 06:45:30.095002 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:30.095017 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:30.095236 (+ 219us) server_negotiation.cc:407] Connection header received
0810 06:45:30.095373 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:30.095381 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:30.095442 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:30.095603 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:30.096424 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.097470 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.098248 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.098690 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.098814 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:30.101419 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:30.101424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:30.101425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:30.101438 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:30.103228 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.103690 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.103693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.103694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.103798 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.104030 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.104031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.104031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.104347 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:30.104536 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:30.104609 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 06:45:30.104763 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":111,"threads_started":1}
I20260810 06:45:30.116621 22705 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:30.117429 22705 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:30.121942 22705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e3eeb8a2da94c589b019531c11c468d: Generated new TSK 0
I20260810 06:45:30.122149 22705 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:45:30.157100 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:32891
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:30.245986 22720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:30.246244 22720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:30.246289 22720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:30.247484 22720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:30.247556 22720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:30.247601 22720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:30.249178 22720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:32891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:30.249457 22720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:30.249694 22720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:30.255059 22725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:30.255097 22726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:30.255055 22728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:30.255097 22720 server_base.cc:1034] running on GCE node
I20260810 06:45:30.255470 22720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:30.255761 22720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:30.256945 22720 hybrid_clock.cc:648] HybridClock initialized: now 1786344330256920 us; error 42 us; skew 500 ppm
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:30.259723 22720 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:45:30.260133 22720 webserver.cc:460] Webserver started at http://127.15.232.193:46543/ using document root <none> and password file <none>
I20260810 06:45:30.260344 22720 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:30.260386 22720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:30.260479 22720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:30.261597 22720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "976a04d864774bc88002b89ecfd730ab"
format_stamp: "Formatted at 2026-08-10 06:45:30 on dist-test-slave-vxj2"
server_key: "20b308bd3e89b7a47a6695a6dc27e16c"
server_key_iv: "6e5fb5de720a0c9df7915fcc1b7a5b60"
server_key_version: "encryptionkey@0"
I20260810 06:45:30.261917 22720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "976a04d864774bc88002b89ecfd730ab"
format_stamp: "Formatted at 2026-08-10 06:45:30 on dist-test-slave-vxj2"
server_key: "20b308bd3e89b7a47a6695a6dc27e16c"
server_key_iv: "6e5fb5de720a0c9df7915fcc1b7a5b60"
server_key_version: "encryptionkey@0"
I20260810 06:45:30.263340 22720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:30.264111 22736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:30.264410 22720 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:30.264495 22720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "976a04d864774bc88002b89ecfd730ab"
format_stamp: "Formatted at 2026-08-10 06:45:30 on dist-test-slave-vxj2"
server_key: "20b308bd3e89b7a47a6695a6dc27e16c"
server_key_iv: "6e5fb5de720a0c9df7915fcc1b7a5b60"
server_key_version: "encryptionkey@0"
I20260810 06:45:30.264590 22720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:30.275259 22720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:30.277472 22720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:30.277590 22720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:30.277774 22720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:30.278107 22720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:30.278141 22720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:30.278164 22720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:30.278211 22720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:30.284375 22720 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:43791
I20260810 06:45:30.284472 22860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:43791 every 8 connection(s)
I20260810 06:45:30.284811 22720 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:30.293319 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 22720
I20260810 06:45:30.293478 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:30.293708 16291 external_mini_cluster.cc:1428] Setting key 0a99229714a39d8e504cbf8cf60dcb46
I20260810 06:45:30.295755 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:30.285710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:32855 (local address 127.15.232.254:32891)
0810 06:45:30.285842 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:30.285845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:30.286028 (+ 183us) server_negotiation.cc:407] Connection header received
0810 06:45:30.287054 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:30.287055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:30.287069 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:30.287131 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:30.287822 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.288361 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.289524 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.289777 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.289875 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:30.292307 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:30.292309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:30.292309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:30.292318 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:30.294087 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.294617 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.294618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.294618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.294690 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.295011 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.295013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.295013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.295170 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:30.295242 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:30.295511 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 06:45:30.295612 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 06:45:30.295846 22863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:30.285627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:32891 (local address 127.15.232.193:32855)
0810 06:45:30.285876 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:30.285934 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:30.286926 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:30.287217 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:30.287220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:30.287497 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:30.287691 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:30.287696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.288471 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.288472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:30.289411 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:30.289414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.289887 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.289888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:30.290019 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:30.290573 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:30.290583 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:30.292199 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:30.294169 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:30.294177 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:30.294179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:30.294532 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:30.294810 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:30.294811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:30.294811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:30.294937 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:30.295242 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:30.295248 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:30.295377 (+ 129us) client_negotiation.cc:769] Sending connection context
0810 06:45:30.295468 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 06:45:30.295607 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":113,"threads_started":1}
I20260810 06:45:30.296443 22861 heartbeater.cc:347] Connected to a master server at 127.15.232.254:32891
I20260810 06:45:30.296578 22861 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:30.296770 22861 heartbeater.cc:511] Master 127.15.232.254:32891 requested a full tablet report, sending...
I20260810 06:45:30.297447 22651 ts_manager.cc:194] Registered new tserver with Master: 976a04d864774bc88002b89ecfd730ab (127.15.232.193:43791)
I20260810 06:45:30.298533 22651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:32855
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:45:30.324761 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:32891
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:30.416088 22866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:30.416306 22866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:30.416332 22866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:30.417596 22866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:30.417642 22866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:30.417670 22866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:30.419253 22866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:32891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:30.419523 22866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:30.419797 22866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:30.425439 22872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:30.425581 22866 server_base.cc:1034] running on GCE node
W20260810 06:45:30.425457 22874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:30.425446 22871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:30.425961 22866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:30.426190 22866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:30.427340 22866 hybrid_clock.cc:648] HybridClock initialized: now 1786344330427317 us; error 50 us; skew 500 ppm
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:30.430187 22866 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:45:30.430650 22866 webserver.cc:460] Webserver started at http://127.15.232.194:40217/ using document root <none> and password file <none>
I20260810 06:45:30.430925 22866 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:30.430998 22866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:30.431151 22866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:30.432242 22866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "ec3c4ef9f7ab4f5b96be43db9b439df2"
format_stamp: "Formatted at 2026-08-10 06:45:30 on dist-test-slave-vxj2"
server_key: "796cf2827d25ccd2f9e519ddb4f8e55c"
server_key_iv: "6be0f268638b9f0666f6badae03b388d"
server_key_version: "encryptionkey@0"
I20260810 06:45:30.432613 22866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "ec3c4ef9f7ab4f5b96be43db9b439df2"
format_stamp: "Formatted at 2026-08-10 06:45:30 on dist-test-slave-vxj2"
server_key: "796cf2827d25ccd2f9e519ddb4f8e55c"
server_key_iv: "6be0f268638b9f0666f6badae03b388d"
server_key_version: "encryptionkey@0"
I20260810 06:45:30.434147 22866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:30.434981 22881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:30.435220 22866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:30.435330 22866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "ec3c4ef9f7ab4f5b96be43db9b439df2"
format_stamp: "Formatted at 2026-08-10 06:45:30 on dist-test-slave-vxj2"
server_key: "796cf2827d25ccd2f9e519ddb4f8e55c"
server_key_iv: "6be0f268638b9f0666f6badae03b388d"
server_key_version: "encryptionkey@0"
I20260810 06:45:30.435407 22866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:30.451078 22866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:30.453560 22866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:30.453742 22866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:30.454031 22866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:30.454384 22866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:30.454440 22866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:30.454483 22866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:30.454525 22866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:30.461102 22866 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:42985
I20260810 06:45:30.461262 23006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:42985 every 8 connection(s)
I20260810 06:45:30.461568 22866 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:30.470809 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 22866
I20260810 06:45:30.470942 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:30.471149 16291 external_mini_cluster.cc:1428] Setting key 5346d8a8570fe6f8d3cf33f79ed2cf76
I20260810 06:45:30.471912 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:30.462404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:42185 (local address 127.15.232.254:32891)
0810 06:45:30.462470 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:30.462472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:30.462629 (+ 157us) server_negotiation.cc:407] Connection header received
0810 06:45:30.463604 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:30.463606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:30.463619 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:30.463701 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:30.464323 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.464927 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.466254 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.466480 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.466554 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:30.468948 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:30.468949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:30.468950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:30.469006 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:30.470352 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.470756 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.470758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.470758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.470829 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.471157 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.471158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.471159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.471310 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:30.471389 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:30.471696 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 06:45:30.471810 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 06:45:30.472123 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:30.462320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:32891 (local address 127.15.232.194:42185)
0810 06:45:30.462491 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:30.462534 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:30.463484 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:30.463760 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:30.463763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:30.464034 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:30.464234 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:30.464239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.465032 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.465033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:30.466145 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:30.466148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.466567 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.466567 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:30.466677 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:30.467184 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:30.467192 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:30.468832 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:30.470432 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:30.470438 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:30.470440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:30.470676 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:30.470952 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:30.470953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:30.470953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:30.471085 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:30.471400 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:30.471407 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:30.471566 (+ 159us) client_negotiation.cc:769] Sending connection context
0810 06:45:30.471656 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 06:45:30.471851 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
I20260810 06:45:30.472579 23007 heartbeater.cc:347] Connected to a master server at 127.15.232.254:32891
I20260810 06:45:30.472690 23007 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:30.472944 23007 heartbeater.cc:511] Master 127.15.232.254:32891 requested a full tablet report, sending...
I20260810 06:45:30.473354 22651 ts_manager.cc:194] Registered new tserver with Master: ec3c4ef9f7ab4f5b96be43db9b439df2 (127.15.232.194:42985)
I20260810 06:45:30.473913 22651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:42185
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:45:30.497913 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:32891
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:30.589293 23013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:30.589519 23013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:30.589545 23013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:30.590832 23013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:30.590875 23013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:30.590904 23013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:30.592516 23013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:32891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:30.592801 23013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:30.593091 23013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:30.598984 23020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:30.599004 23019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:30.599061 23022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:30.599431 23013 server_base.cc:1034] running on GCE node
I20260810 06:45:30.599710 23013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:30.599982 23013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:30.601203 23013 hybrid_clock.cc:648] HybridClock initialized: now 1786344330601173 us; error 60 us; skew 500 ppm
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:30.607574 23013 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:45:30.608115 23013 webserver.cc:460] Webserver started at http://127.15.232.195:39613/ using document root <none> and password file <none>
I20260810 06:45:30.608361 23013 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:30.608443 23013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:30.608601 23013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:30.609778 23013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "a8f5c37a0a134b6c9dd4d0555e935f8e"
format_stamp: "Formatted at 2026-08-10 06:45:30 on dist-test-slave-vxj2"
server_key: "dac30d773df38ed3ce4376e49d17d83e"
server_key_iv: "8181eab3acedb8c5785a2bf13a7d5672"
server_key_version: "encryptionkey@0"
I20260810 06:45:30.610160 23013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "a8f5c37a0a134b6c9dd4d0555e935f8e"
format_stamp: "Formatted at 2026-08-10 06:45:30 on dist-test-slave-vxj2"
server_key: "dac30d773df38ed3ce4376e49d17d83e"
server_key_iv: "8181eab3acedb8c5785a2bf13a7d5672"
server_key_version: "encryptionkey@0"
I20260810 06:45:30.611627 23013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:30.612470 23029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:30.612715 23013 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:30.612830 23013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "a8f5c37a0a134b6c9dd4d0555e935f8e"
format_stamp: "Formatted at 2026-08-10 06:45:30 on dist-test-slave-vxj2"
server_key: "dac30d773df38ed3ce4376e49d17d83e"
server_key_iv: "8181eab3acedb8c5785a2bf13a7d5672"
server_key_version: "encryptionkey@0"
I20260810 06:45:30.612911 23013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:30.635057 23013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:30.637434 23013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:30.637568 23013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:30.637801 23013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:30.638180 23013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:30.638219 23013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:30.638247 23013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:30.638259 23013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:30.646183 23013 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:35533
I20260810 06:45:30.646268 23152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:35533 every 8 connection(s)
I20260810 06:45:30.646658 23013 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:30.654134 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 23013
I20260810 06:45:30.654251 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:30.654465 16291 external_mini_cluster.cc:1428] Setting key f0e9275d17d9a4f9e4695cceb73df214
I20260810 06:45:30.657994 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:30.647568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:54393 (local address 127.15.232.254:32891)
0810 06:45:30.647665 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:30.647668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:30.647917 (+ 249us) server_negotiation.cc:407] Connection header received
0810 06:45:30.649103 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:30.649105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:30.649119 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:30.649206 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:30.649946 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.650514 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.651677 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.651914 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.652011 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:30.654601 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:30.654602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:30.654603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:30.654613 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:30.656304 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.656824 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.656825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.656826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.656897 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.657225 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.657226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.657227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.657374 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:30.657463 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:30.657790 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 06:45:30.657887 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:45:30.658231 23155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:30.647532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:32891 (local address 127.15.232.195:54393)
0810 06:45:30.647738 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:30.647825 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:30.648963 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:30.649297 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:30.649301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:30.649599 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:30.649834 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:30.649839 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.650625 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.650626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:30.651573 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:30.651575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.652024 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.652025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:30.652164 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:30.652792 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:30.652802 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:30.654480 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:30.656423 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:30.656431 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:30.656433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:30.656700 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:30.656973 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:30.656975 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:30.656975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:30.657108 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:30.657455 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:30.657463 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:30.657634 (+ 171us) client_negotiation.cc:769] Sending connection context
0810 06:45:30.657771 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 06:45:30.657962 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260810 06:45:30.658710 23153 heartbeater.cc:347] Connected to a master server at 127.15.232.254:32891
I20260810 06:45:30.658829 23153 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:30.658998 23153 heartbeater.cc:511] Master 127.15.232.254:32891 requested a full tablet report, sending...
I20260810 06:45:30.659401 22651 ts_manager.cc:194] Registered new tserver with Master: a8f5c37a0a134b6c9dd4d0555e935f8e (127.15.232.195:35533)
I20260810 06:45:30.659785 22651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:54393
I20260810 06:45:30.666613 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/test-admin.keytab.
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:30.686666 16291 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 06:45:30.688501 23160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:30.687661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43802 (local address 127.15.232.193:43791)
0810 06:45:30.687856 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:30.687862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:30.687873 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:30.687910 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:30.687913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:30.687926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:30.688015 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:30.688417 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43802: BlockingRecv error: recv got EOF from 127.0.0.1:43802 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
W20260810 06:45:30.688637 16291 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:30.689843 16291 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:30.690862 16291 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.193@KRBTEST.COM
I20260810 06:45:30.699594 23160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:30.691545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43812 (local address 127.15.232.193:43791)
0810 06:45:30.691617 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:30.691619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:30.691646 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:45:30.691730 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:30.691731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:30.691749 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:30.691815 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:30.692242 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.693030 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.693904 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.694233 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.694318 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:30.696541 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:30.696549 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:30.696550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:30.696557 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:30.698378 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.698815 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.698818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.698819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.698884 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.699085 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.699085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.699086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.699231 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:30.699349 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:30.699430 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 06:45:30.699508 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 06:45:30.702188 23160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:30.701524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43826 (local address 127.15.232.193:43791)
0810 06:45:30.701585 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:30.701586 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:30.701650 (+ 64us) server_negotiation.cc:407] Connection header received
0810 06:45:30.701727 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:30.701727 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:30.701739 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:30.701789 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:30.702128 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43826: BlockingRecv error: recv got EOF from 127.0.0.1:43826 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 06:45:30.702240 16291 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:30.703236 16291 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:30.704200 16291 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.193@KRBTEST.COM
I20260810 06:45:30.712239 23160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:30.704828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43832 (local address 127.15.232.193:43791)
0810 06:45:30.704885 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:30.704887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:30.704977 (+ 90us) server_negotiation.cc:407] Connection header received
0810 06:45:30.705058 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:30.705059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:30.705069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:30.705120 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:30.705555 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.706050 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.706774 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.706989 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.707069 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:30.709450 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:30.709451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:30.709452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:30.709460 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:30.710905 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.711346 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.711348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.711348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.711406 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.711634 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.711635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.711635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.711754 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:30.711868 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:30.712023 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 06:45:30.712120 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260810 06:45:30.714208 23160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:30.713571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.15.232.193:43791)
0810 06:45:30.713637 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:30.713639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:30.713695 (+ 56us) server_negotiation.cc:407] Connection header received
0810 06:45:30.713778 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:30.713779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:30.713788 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:30.713834 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:30.714145 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43834: BlockingRecv error: recv got EOF from 127.0.0.1:43834 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 06:45:30.714267 16291 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344303981681-16291-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:30.715524 16291 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:30.716580 16291 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[22598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.193@KRBTEST.COM
I20260810 06:45:30.724660 23160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:30.717224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43840 (local address 127.15.232.193:43791)
0810 06:45:30.717269 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:30.717271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:30.717330 (+ 59us) server_negotiation.cc:407] Connection header received
0810 06:45:30.717411 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:30.717413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:30.717423 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:30.717492 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:30.717993 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.718548 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.719311 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:30.719522 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:30.719614 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:30.721908 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:30.721910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:30.721910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:30.721919 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:30.723295 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.723700 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.723702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.723702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.723774 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:30.724030 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:30.724031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:30.724031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:30.724151 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:30.724317 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:30.724459 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 06:45:30.724545 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:45:30.725440 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 22720
I20260810 06:45:30.733349 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 22866
I20260810 06:45:30.741330 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 23013
I20260810 06:45:30.748718 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 22612
2026-08-10T06:45:30Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2047 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[23173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[23173](info): set up 2 sockets
Aug 10 06:45:30 dist-test-slave-vxj2 krb5kdc[23173](info): commencing operation
krb5kdc: starting...
W20260810 06:45:31.925180 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:32Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.254.keytab.
Entry for principal oryx/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.254.keytab.
I20260810 06:45:32.053635 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:41477
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.254.keytab
--principal=oryx/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:41477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:32.135317 23186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:32.135536 23186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:32.135560 23186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:32.136757 23186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:32.136801 23186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:32.136814 23186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:32.136826 23186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:32.136839 23186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:32.138370 23186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:41477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.254.keytab
--principal=oryx/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:41477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:32.138630 23186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:32.138888 23186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:32.143852 23191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:32.143870 23192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:32.143985 23194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:32.144178 23186 server_base.cc:1034] running on GCE node
I20260810 06:45:32.144387 23186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:32.144665 23186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:32.145802 23186 hybrid_clock.cc:648] HybridClock initialized: now 1786344332145784 us; error 39 us; skew 500 ppm
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:32.148386 23186 init.cc:377] Logged in from keytab as oryx/127.15.232.254@KRBTEST.COM (short username oryx)
I20260810 06:45:32.148794 23186 webserver.cc:460] Webserver started at http://127.15.232.254:45101/ using document root <none> and password file <none>
I20260810 06:45:32.148999 23186 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:32.149040 23186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:32.149128 23186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:32.150280 23186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "1d4688810d5e4109b0382f56df98528b"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "c46d24ce2e9292d0768bbbbae2736c55"
server_key_iv: "c8383d44c197dab3b909244830b3bdfb"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.150595 23186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "1d4688810d5e4109b0382f56df98528b"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "c46d24ce2e9292d0768bbbbae2736c55"
server_key_iv: "c8383d44c197dab3b909244830b3bdfb"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.151906 23186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:45:32.152633 23201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.152827 23186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:32.152916 23186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "1d4688810d5e4109b0382f56df98528b"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "c46d24ce2e9292d0768bbbbae2736c55"
server_key_iv: "c8383d44c197dab3b909244830b3bdfb"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.152971 23186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:32.165792 23186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:32.168166 23186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:32.168285 23186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:32.172679 23186 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:41477
I20260810 06:45:32.172734 23260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:41477 every 8 connection(s)
I20260810 06:45:32.173228 23186 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:32.174341 23261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:32.176708 23261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b: Bootstrap starting.
I20260810 06:45:32.177345 23261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:32.177636 23261 log.cc:824] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:32.178427 23261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b: No bootstrap required, opened a new log
I20260810 06:45:32.178717 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 23186
I20260810 06:45:32.178850 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:32.179088 16291 external_mini_cluster.cc:1428] Setting key ee470ee404b8b8fa5ca19190c859467f
I20260810 06:45:32.180884 23261 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } }
I20260810 06:45:32.181090 23261 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:32.181131 23261 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d4688810d5e4109b0382f56df98528b, State: Initialized, Role: FOLLOWER
I20260810 06:45:32.181305 23261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } }
I20260810 06:45:32.181423 23261 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:32.181470 23261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:32.181524 23261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:32.182149 23261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } }
I20260810 06:45:32.182276 23261 leader_election.cc:302] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d4688810d5e4109b0382f56df98528b; no voters:
I20260810 06:45:32.182487 23261 leader_election.cc:288] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:32.182509 23267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:32.182634 23267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 1 LEADER]: Becoming Leader. State: Replica: 1d4688810d5e4109b0382f56df98528b, State: Running, Role: LEADER
I20260810 06:45:32.182722 23267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } }
I20260810 06:45:32.182783 23261 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:32.183072 23268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } }
I20260810 06:45:32.183163 23268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:32.183072 23269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d4688810d5e4109b0382f56df98528b. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } }
I20260810 06:45:32.183394 23269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: This master's current role is: LEADER
W20260810 06:45:32.184732 23285 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:45:32.184787 23285 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:45:32.184835 23273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:32.185026 23273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
I20260810 06:45:32.186880 23273 catalog_manager.cc:1347] Generated new cluster ID: 668e327a50224196a59c433bb74e826b
I20260810 06:45:32.186930 23273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:32.190740 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.179647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40778 (local address 127.15.232.254:41477)
0810 06:45:32.179848 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:32.179861 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:32.179888 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:45:32.180319 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:32.180325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:32.180378 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:32.180490 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:32.181350 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.182764 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.183861 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.184421 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.184527 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.187319 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:32.187323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:32.187323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:32.187336 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:32.189108 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.189564 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.189574 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.189574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.189661 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.189881 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.189883 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.189883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.190123 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:32.190312 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:32.190381 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 06:45:32.190524 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":117,"threads_started":1}
I20260810 06:45:32.202559 23273 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:32.203107 23273 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:32.208877 23273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b: Generated new TSK 0
I20260810 06:45:32.209082 23273 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab.
Entry for principal oryx/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab.
I20260810 06:45:32.238233 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41477
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab
--principal=oryx/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:32.320596 23290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:32.320812 23290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:32.320835 23290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:32.322017 23290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:32.322062 23290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:32.322088 23290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:32.323623 23290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab
--principal=oryx/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:32.323869 23290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:32.324117 23290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:32.329223 23303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:32.329259 23297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:32.329249 23298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:32.329480 23290 server_base.cc:1034] running on GCE node
I20260810 06:45:32.329639 23290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:32.329867 23290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:32.331015 23290 hybrid_clock.cc:648] HybridClock initialized: now 1786344332331003 us; error 33 us; skew 500 ppm
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:32.333622 23290 init.cc:377] Logged in from keytab as oryx/127.15.232.193@KRBTEST.COM (short username oryx)
I20260810 06:45:32.334079 23290 webserver.cc:460] Webserver started at http://127.15.232.193:34921/ using document root <none> and password file <none>
I20260810 06:45:32.334321 23290 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:32.334367 23290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:32.334501 23290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:32.335604 23290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "c8fcb6da6ba24b09a602df3b23a4c521"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "9ccfeb9fef067530f9e097acbdcac39d"
server_key_iv: "65fe8cf97e28b9e018d97cb8a595eb2a"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.335978 23290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "c8fcb6da6ba24b09a602df3b23a4c521"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "9ccfeb9fef067530f9e097acbdcac39d"
server_key_iv: "65fe8cf97e28b9e018d97cb8a595eb2a"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.337472 23290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:45:32.338261 23310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.338554 23290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:32.338663 23290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "c8fcb6da6ba24b09a602df3b23a4c521"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "9ccfeb9fef067530f9e097acbdcac39d"
server_key_iv: "65fe8cf97e28b9e018d97cb8a595eb2a"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.338738 23290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:32.371219 23290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:32.373423 23290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:32.373610 23290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:32.373839 23290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:32.374169 23290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:32.374203 23290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.374248 23290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:32.374274 23290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.380261 23290 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:38905
I20260810 06:45:32.380336 23432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:38905 every 8 connection(s)
I20260810 06:45:32.380692 23290 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:32.383844 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 23290
I20260810 06:45:32.383967 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:32.384155 16291 external_mini_cluster.cc:1428] Setting key b6e5c1b5c52c5f1ad3cabd8697e0e9b7
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
I20260810 06:45:32.392355 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.381524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:56151 (local address 127.15.232.254:41477)
0810 06:45:32.381619 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:32.381622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:32.381818 (+ 196us) server_negotiation.cc:407] Connection header received
0810 06:45:32.382385 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:32.382387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:32.382399 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:32.382446 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:32.383567 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.384335 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.385995 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.386197 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.386258 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.388632 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:32.388633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:32.388634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:32.388643 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:32.390642 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.391172 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.391173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.391174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.391242 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.391550 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.391551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.391551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.391703 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:32.391784 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:32.392121 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 06:45:32.392229 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:45:32.392539 23437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.381512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.193:56151)
0810 06:45:32.381684 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:32.381728 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:32.382258 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:32.382522 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:32.382526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:32.383209 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:32.383465 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.383469 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.384503 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.384504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.385902 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.385906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.386288 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.386290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.386453 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.387093 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:32.387101 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:32.388523 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:32.390803 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.390811 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.390813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.391083 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.391344 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.391344 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.391345 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.391461 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.391814 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:32.391822 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:32.391965 (+ 143us) client_negotiation.cc:769] Sending connection context
0810 06:45:32.392075 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 06:45:32.392217 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260810 06:45:32.393129 23433 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41477
I20260810 06:45:32.393308 23433 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.15.232.194@KRBTEST.COM; defaulting to no policy
I20260810 06:45:32.393561 23433 heartbeater.cc:511] Master 127.15.232.254:41477 requested a full tablet report, sending...
I20260810 06:45:32.394212 23220 ts_manager.cc:194] Registered new tserver with Master: c8fcb6da6ba24b09a602df3b23a4c521 (127.15.232.193:38905)
I20260810 06:45:32.394882 23220 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:56151
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab.
Entry for principal oryx/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab.
I20260810 06:45:32.412528 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41477
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab
--principal=oryx/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:32.503101 23440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:32.503328 23440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:32.503353 23440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:32.504499 23440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:32.504539 23440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:32.504565 23440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:32.506002 23440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab
--principal=oryx/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:32.506217 23440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:32.506398 23440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:32.511693 23446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:32.511821 23447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:32.511698 23449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:32.511993 23440 server_base.cc:1034] running on GCE node
I20260810 06:45:32.512266 23440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:32.512526 23440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:32.513705 23440 hybrid_clock.cc:648] HybridClock initialized: now 1786344332513696 us; error 47 us; skew 500 ppm
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:32.516381 23440 init.cc:377] Logged in from keytab as oryx/127.15.232.194@KRBTEST.COM (short username oryx)
I20260810 06:45:32.516855 23440 webserver.cc:460] Webserver started at http://127.15.232.194:34349/ using document root <none> and password file <none>
I20260810 06:45:32.517095 23440 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:32.517185 23440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:32.517371 23440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:32.518445 23440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "4656f0cf82604745b73b24e68adce362"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "01c4302bc12a162c6c3d32e55a6c0766"
server_key_iv: "aa3180cec757e05a34a79dc3e16efa30"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.518824 23440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "4656f0cf82604745b73b24e68adce362"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "01c4302bc12a162c6c3d32e55a6c0766"
server_key_iv: "aa3180cec757e05a34a79dc3e16efa30"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.520228 23440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:32.521013 23456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.521256 23440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:32.521395 23440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "4656f0cf82604745b73b24e68adce362"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "01c4302bc12a162c6c3d32e55a6c0766"
server_key_iv: "aa3180cec757e05a34a79dc3e16efa30"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.521477 23440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:32.548393 23440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:32.550518 23440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:32.550715 23440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:32.551008 23440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:32.551363 23440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:32.551421 23440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.551457 23440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:32.551505 23440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.557822 23440 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:33315
I20260810 06:45:32.557866 23580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:33315 every 8 connection(s)
I20260810 06:45:32.558244 23440 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:32.558914 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 23440
I20260810 06:45:32.559032 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:32.559258 16291 external_mini_cluster.cc:1428] Setting key 2bee1a01eb003c06461718cf70462d4c
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.15.232.195@KRBTEST.COM; defaulting to no policy
I20260810 06:45:32.568415 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.559073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:41457 (local address 127.15.232.254:41477)
0810 06:45:32.559167 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:32.559170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:32.559395 (+ 225us) server_negotiation.cc:407] Connection header received
0810 06:45:32.560035 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:32.560037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:32.560049 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:32.560086 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:32.561178 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.561688 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.562735 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.562960 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.563052 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.565376 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:32.565378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:32.565378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:32.565387 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:32.566835 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.567350 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.567351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.567352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.567418 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.567687 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.567688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.567689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.567811 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:32.567894 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:32.568225 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 06:45:32.568317 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:45:32.568631 23583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.559059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.194:41457)
0810 06:45:32.559239 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:32.559282 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:32.559943 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:32.560169 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:32.560174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:32.560810 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:32.561034 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.561039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.561803 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.561804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.562635 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.562637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.563066 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.563067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.563208 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.563799 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:32.563808 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:32.565262 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:32.566949 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.566957 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.566959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.567252 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.567486 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.567487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.567488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.567615 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.567884 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:32.567892 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:32.568042 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 06:45:32.568196 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 06:45:32.568380 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260810 06:45:32.569140 23581 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41477
I20260810 06:45:32.569288 23581 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:32.569520 23581 heartbeater.cc:511] Master 127.15.232.254:41477 requested a full tablet report, sending...
I20260810 06:45:32.569926 23220 ts_manager.cc:194] Registered new tserver with Master: 4656f0cf82604745b73b24e68adce362 (127.15.232.194:33315)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.195@KRBTEST.COM" created.
I20260810 06:45:32.570374 23220 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:41457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab.
Entry for principal oryx/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab.
I20260810 06:45:32.588657 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41477
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab
--principal=oryx/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:32.673884 23586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:32.674098 23586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:32.674124 23586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:32.675310 23586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:32.675355 23586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:32.675382 23586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:32.676864 23586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab
--principal=oryx/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:32.677111 23586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:32.677371 23586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:32.682821 23592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:32.682804 23594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:32.682804 23591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:32.683573 23586 server_base.cc:1034] running on GCE node
I20260810 06:45:32.683847 23586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:32.684146 23586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:32.685305 23586 hybrid_clock.cc:648] HybridClock initialized: now 1786344332685289 us; error 37 us; skew 500 ppm
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:32.687959 23586 init.cc:377] Logged in from keytab as oryx/127.15.232.195@KRBTEST.COM (short username oryx)
I20260810 06:45:32.688438 23586 webserver.cc:460] Webserver started at http://127.15.232.195:36451/ using document root <none> and password file <none>
I20260810 06:45:32.688628 23586 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:32.688668 23586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:32.688759 23586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:32.689845 23586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "926b62eab226485689e4c322ef7e4a3d"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "59dd1d8c27069b792965b6f2f9a5b24c"
server_key_iv: "1fd36795246df6c93c83973f77a76028"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.690171 23586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "926b62eab226485689e4c322ef7e4a3d"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "59dd1d8c27069b792965b6f2f9a5b24c"
server_key_iv: "1fd36795246df6c93c83973f77a76028"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.691504 23586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:32.692308 23602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.692554 23586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:32.692646 23586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "926b62eab226485689e4c322ef7e4a3d"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "59dd1d8c27069b792965b6f2f9a5b24c"
server_key_iv: "1fd36795246df6c93c83973f77a76028"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.692706 23586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:32.709543 23586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:32.711937 23586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:32.712059 23586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:32.712266 23586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:32.712550 23586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:32.712584 23586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.712606 23586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:32.712618 23586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.718791 23586 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:38029
I20260810 06:45:32.718880 23722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:38029 every 8 connection(s)
I20260810 06:45:32.719213 23586 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:32.724308 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 23586
I20260810 06:45:32.724417 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:32.724628 16291 external_mini_cluster.cc:1428] Setting key 73f737a60d2cb153034f9cd8d38f9866
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
I20260810 06:45:32.729658 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.719996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:44377 (local address 127.15.232.254:41477)
0810 06:45:32.720057 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:32.720059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:32.720261 (+ 202us) server_negotiation.cc:407] Connection header received
0810 06:45:32.720745 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:32.720746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:32.720755 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:32.720870 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:32.721929 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.722515 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.723552 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.723765 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.723831 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.726588 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:32.726589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:32.726589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:32.726597 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:32.728231 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.728624 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.728625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.728626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.728687 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.728942 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.728943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.728943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.729060 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:32.729118 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:32.729498 (+ 380us) server_negotiation.cc:299] Negotiation successful
0810 06:45:32.729586 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:45:32.729729 23726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.719959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.195:44377)
0810 06:45:32.720122 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:32.720164 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:32.720661 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:32.720931 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:32.720935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:32.721602 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:32.721803 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.721808 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.722609 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.722610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.723447 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.723449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.723851 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.723852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.723958 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.724578 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:32.724587 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:32.726497 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:32.728308 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.728314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.728316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.728529 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.728758 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.728758 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.728758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.728856 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.729119 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:32.729124 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:32.729308 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 06:45:32.729400 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 06:45:32.729531 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260810 06:45:32.730275 23724 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41477
I20260810 06:45:32.730381 23724 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:32.730594 23724 heartbeater.cc:511] Master 127.15.232.254:41477 requested a full tablet report, sending...
I20260810 06:45:32.731003 23220 ts_manager.cc:194] Registered new tserver with Master: 926b62eab226485689e4c322ef7e4a3d (127.15.232.195:38029)
I20260810 06:45:32.731395 23220 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:44377
I20260810 06:45:32.736892 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.232.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.253.keytab.
Entry for principal oryx/127.15.232.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.253.keytab.
I20260810 06:45:32.764937 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.253:34383
--webserver_interface=127.15.232.253
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.253.keytab
--principal=oryx/127.15.232.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:41477,127.15.232.253:34383
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:32.852517 23729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:32.852787 23729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:32.852828 23729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:32.854063 23729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:32.854144 23729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:32.854168 23729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:32.854192 23729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:32.854219 23729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:32.855778 23729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:41477,127.15.232.253:34383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.253.keytab
--principal=oryx/127.15.232.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.253:34383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.232.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
W20260810 06:45:32.855926 23729 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.232.254:41477,127.15.232.253:34383'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 06:45:32.856117 23729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:32.856369 23729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:32.861330 23734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:32.861334 23735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:32.861462 23729 server_base.cc:1034] running on GCE node
W20260810 06:45:32.861338 23737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:32.861999 23729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:32.862321 23729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:32.863502 23729 hybrid_clock.cc:648] HybridClock initialized: now 1786344332863485 us; error 45 us; skew 500 ppm
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:32.866524 23729 init.cc:377] Logged in from keytab as oryx/127.15.232.253@KRBTEST.COM (short username oryx)
I20260810 06:45:32.866971 23729 webserver.cc:460] Webserver started at http://127.15.232.253:39815/ using document root <none> and password file <none>
I20260810 06:45:32.867168 23729 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:32.867213 23729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:32.867327 23729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:32.868450 23729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data/instance:
uuid: "6a377bc7a58e46e28aa826fa3f084efb"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "ce786e86743aa0788f2eabff7eb24ae5"
server_key_iv: "6613e073b2a3ca77d040a264e0e0e866"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.868811 23729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/wal/instance:
uuid: "6a377bc7a58e46e28aa826fa3f084efb"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "ce786e86743aa0788f2eabff7eb24ae5"
server_key_iv: "6613e073b2a3ca77d040a264e0e0e866"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.870299 23729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:45:32.871186 23747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.871381 23729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:32.871460 23729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/wal
uuid: "6a377bc7a58e46e28aa826fa3f084efb"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "ce786e86743aa0788f2eabff7eb24ae5"
server_key_iv: "6613e073b2a3ca77d040a264e0e0e866"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.871517 23729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:32.890913 23729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:32.893605 23729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:32.893733 23729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:32.898255 23729 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.253:34383
I20260810 06:45:32.898288 23807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.253:34383 every 8 connection(s)
I20260810 06:45:32.898689 23729 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
I20260810 06:45:32.899758 23808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:32.901005 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 23729
I20260810 06:45:32.901692 23808 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.253@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.232.253@KRBTEST.COM
I20260810 06:45:32.913592 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.902519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.15.232.254:41477)
0810 06:45:32.902593 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:32.902596 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:32.902733 (+ 137us) server_negotiation.cc:407] Connection header received
0810 06:45:32.902982 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:32.902983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:32.902999 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:32.903064 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:32.903858 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.904724 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.906296 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.906617 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.906704 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.909842 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:32.909844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:32.909844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:32.909853 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:32.911586 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.912208 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.912210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.912210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.912287 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.912618 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.912619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.912619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.912869 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:32.912952 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:32.913372 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 06:45:32.913463 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:45:32.913843 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.902452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:40794)
0810 06:45:32.902617 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:32.902658 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:32.902811 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:32.903104 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:32.903107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:32.903489 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:32.903731 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.903737 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.904872 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.904873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.906166 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.906169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.906782 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.906783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.906914 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.907695 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:32.907775 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:32.909663 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:32.911683 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.911689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.911691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.912100 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.912378 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.912379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.912379 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.912517 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.912956 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:32.912961 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:32.913129 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 06:45:32.913329 (+ 200us) client_negotiation.cc:240] Negotiation successful
0810 06:45:32.913558 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":5,"thread_start_us":84,"threads_started":1}
I20260810 06:45:32.914162 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.901878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.15.232.253:34383)
0810 06:45:32.902077 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:32.902087 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:32.902123 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:45:32.902813 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:32.902821 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:32.902845 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:32.902972 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:32.903784 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.905059 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.906296 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.906825 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.906912 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.910281 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:32.910290 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:32.910291 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:32.910306 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:32.912308 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.912873 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.912876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.912877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.912959 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.913250 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.913251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.913251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.913429 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:32.913609 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:32.913855 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 06:45:32.914094 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
I20260810 06:45:32.914811 23808 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.253@KRBTEST.COM for oryx/127.15.232.253@KRBTEST.COM
I20260810 06:45:32.921321 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.915129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:34383 (local address 127.0.0.1:50854)
0810 06:45:32.915184 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:32.915195 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:32.915238 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:32.915383 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:32.915384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:32.915532 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:32.915577 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.915578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.916278 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.916279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.916794 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.916795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.917209 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.917210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.917294 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.917620 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:32.917624 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:32.918784 (+ 1160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:32.920344 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.920344 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.920345 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.920516 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.920696 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.920696 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.920697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.920763 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.920989 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:32.920990 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:32.921026 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:45:32.921082 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 06:45:32.921225 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 06:45:32.921365 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.915204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50854 (local address 127.15.232.253:34383)
0810 06:45:32.915240 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:32.915242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:32.915251 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:32.915311 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:32.915312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:32.915321 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:32.915379 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:32.915661 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.916187 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.916881 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.917109 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.917215 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.918875 (+ 1660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:32.918876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:32.918877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:32.918882 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:32.920265 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.920583 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.920584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.920584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.920633 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.920821 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.920821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.920822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.920931 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:32.920980 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:32.921096 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 06:45:32.921273 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":19}
I20260810 06:45:32.922700 23808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: Bootstrap starting.
I20260810 06:45:32.923533 23808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:32.923877 23808 log.cc:824] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:32.924649 23808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: No bootstrap required, opened a new log
I20260810 06:45:32.926255 23808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } }
I20260810 06:45:32.926383 23808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:32.926404 23808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a377bc7a58e46e28aa826fa3f084efb, State: Initialized, Role: FOLLOWER
I20260810 06:45:32.926608 23808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } }
I20260810 06:45:32.926947 23808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:32.926935 23814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } } }
I20260810 06:45:32.927050 23814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:45:32.928890 23827 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:45:32.928953 23827 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:45:32.930838 23729 master_runner.cc:428] Detected that this master 6a377bc7a58e46e28aa826fa3f084efb is joining an existing cluster
I20260810 06:45:32.930929 23729 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.232.253:34383
I20260810 06:45:32.939508 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.932085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40800 (local address 127.15.232.254:41477)
0810 06:45:32.932131 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:32.932133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:32.932691 (+ 558us) server_negotiation.cc:407] Connection header received
0810 06:45:32.932824 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:32.932825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:32.932838 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:32.932891 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:32.933319 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.934123 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.934973 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.935274 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.935345 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.936486 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:32.936487 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:32.936488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:32.936495 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:32.938093 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.938539 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.938541 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.938541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.938608 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.938841 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.938843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.938843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.939006 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:32.939085 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:32.939274 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 06:45:32.939383 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:45:32.939860 23834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.932012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:40800)
0810 06:45:32.932605 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:32.932618 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:32.932687 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:32.932937 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:32.932938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:32.933169 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:32.933236 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.933238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.934214 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.934215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.934841 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.934843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.935376 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.935377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.935448 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.935986 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:32.935991 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:32.936390 (+ 399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:32.938188 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.938188 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.938189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.938458 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.938690 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.938691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.938691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.938763 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.939088 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:32.939089 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:32.939137 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:45:32.939626 (+ 489us) client_negotiation.cc:240] Negotiation successful
0810 06:45:32.939757 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":95,"threads_started":1}
I20260810 06:45:32.940614 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.932012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:34383 (local address 127.0.0.1:50860)
0810 06:45:32.932616 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:32.932626 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:32.932687 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:32.932845 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:32.932852 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:32.933117 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:32.933236 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.933238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.935078 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.935079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.935846 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.935849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.936280 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.936281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.936352 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.936785 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:32.936790 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:32.937145 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:32.939136 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.939137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.939137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.939303 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.939497 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.939498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.939498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.939568 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.939870 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:32.939871 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:32.939903 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:45:32.940377 (+ 474us) client_negotiation.cc:240] Negotiation successful
0810 06:45:32.940487 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":2,"thread_start_us":532,"threads_started":1}
I20260810 06:45:32.940659 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.932196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50860 (local address 127.15.232.253:34383)
0810 06:45:32.932252 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:32.932254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:32.932711 (+ 457us) server_negotiation.cc:407] Connection header received
0810 06:45:32.932738 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:32.932739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:32.932750 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:32.932832 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:32.934235 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.934990 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.935911 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.936187 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.936260 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.937468 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:32.937468 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:32.937469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:32.937474 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:32.939079 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.939376 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.939377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.939377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.939433 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.939643 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.939644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.939644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.939779 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:32.939817 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:32.940409 (+ 592us) server_negotiation.cc:299] Negotiation successful
0810 06:45:32.940538 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":37}
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.254@KRBTEST.COM for oryx/127.15.232.253@KRBTEST.COM
I20260810 06:45:32.950387 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.942466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50874 (local address 127.15.232.253:34383)
0810 06:45:32.942507 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:32.942509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:32.942656 (+ 147us) server_negotiation.cc:407] Connection header received
0810 06:45:32.942809 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:32.942810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:32.942819 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:32.942883 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:32.943392 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.943816 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.944955 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.945135 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.945240 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.947349 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:32.947351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:32.947351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:32.947358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:32.948897 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.949470 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.949471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.949472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.949539 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.949780 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.949781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.949781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.949910 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:32.949989 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:32.950195 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 06:45:32.950282 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:45:32.950451 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.942381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:34383 (local address 127.0.0.1:50874)
0810 06:45:32.942541 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:32.942594 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:32.942676 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:32.942905 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:32.942911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:32.943254 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:32.943313 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.943315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.943925 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.943926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.944873 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.944875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.945240 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.945241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.945325 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.945723 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:32.945728 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:32.947261 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:32.949042 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.949051 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.949054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.949372 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.949612 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.949613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.949613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.949714 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.949994 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:32.950002 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:32.950066 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 06:45:32.950166 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:45:32.950316 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260810 06:45:32.951074 23220 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.232.253:34383: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb"
member_type: NON_VOTER
last_known_addr {
host: "127.15.232.253"
port: 34383
}
attrs {
promote: true
}
}
dest_uuid: "1d4688810d5e4109b0382f56df98528b"
cas_config_opid_index: -1
I20260810 06:45:32.951450 23220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } }
I20260810 06:45:32.952023 23837 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6a377bc7a58e46e28aa826fa3f084efb (127.15.232.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } }
I20260810 06:45:32.952400 23778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:32.953303 23837 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.232.253:34383
I20260810 06:45:32.953402 23837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } }
I20260810 06:45:32.953480 23837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:32.953485 23778 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 FOLLOWER]: Refusing update from remote peer 1d4688810d5e4109b0382f56df98528b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 06:45:32.953848 23836 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:32.954077 23729 master.cc:537] Master@127.15.232.253:34383 shutting down...
W20260810 06:45:32.954960 23204 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260810 06:45:32.955505 23204 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b -> Peer 6a377bc7a58e46e28aa826fa3f084efb (127.15.232.253:34383): Couldn't send request to peer 6a377bc7a58e46e28aa826fa3f084efb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 06:45:32.957993 23729 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:32.958093 23729 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:32.958134 23729 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: stopping tablet replica
I20260810 06:45:32.971684 23729 master.cc:552] Master@127.15.232.253:34383 shutdown complete.
I20260810 06:45:32.971742 23729 master_runner.cc:305] Clearing existing system tablet
I20260810 06:45:32.971906 23729 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:32.971956 23729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:32.972625 23729 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.001s
I20260810 06:45:32.973140 23841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.973326 23729 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:32.973383 23729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.973402 23729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/wal
uuid: "6a377bc7a58e46e28aa826fa3f084efb"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "ce786e86743aa0788f2eabff7eb24ae5"
server_key_iv: "6613e073b2a3ca77d040a264e0e0e866"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.974193 23729 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:32.975937 23729 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 06:45:32.976002 23729 log.cc:1196] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 06:45:32.976271 23729 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: Deleting consensus metadata
I20260810 06:45:32.976413 23729 master_runner.cc:315] Copying system tablet from 127.15.232.254:41477
I20260810 06:45:32.977147 23729 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: tablet copy: Beginning tablet copy session from remote peer at address 127.15.232.254:41477
I20260810 06:45:32.984091 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.977644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40814 (local address 127.15.232.254:41477)
0810 06:45:32.977697 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:32.977699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:32.977800 (+ 101us) server_negotiation.cc:407] Connection header received
0810 06:45:32.977865 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:32.977866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:32.977876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:32.977931 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:32.978336 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.978821 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.979936 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.980130 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.980199 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.981315 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:32.981316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:32.981316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:32.981323 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:32.982803 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.983203 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.983204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.983204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.983274 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:32.983498 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:32.983499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:32.983499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:32.983659 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:32.983720 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:32.983918 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 06:45:32.983999 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 06:45:32.984139 23848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:32.977533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:40814)
0810 06:45:32.977694 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:32.977707 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:32.977790 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:32.977960 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:32.977961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:32.978186 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:32.978256 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.978258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.978940 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.978941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.979824 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:32.979827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:32.980213 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:32.980214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:32.980286 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:32.980767 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:32.980772 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:32.981219 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:32.982889 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.982890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.982891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.983123 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.983353 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:32.983354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:32.983354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:32.983428 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:32.983725 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:32.983726 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:32.983774 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:45:32.983885 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 06:45:32.984000 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260810 06:45:32.984491 23242 tablet_copy_service.cc:140] P 1d4688810d5e4109b0382f56df98528b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6a377bc7a58e46e28aa826fa3f084efb ({username='oryx', principal='oryx/127.15.232.253@KRBTEST.COM'} at 127.0.0.1:40814)
I20260810 06:45:32.984642 23242 tablet_copy_service.cc:161] P 1d4688810d5e4109b0382f56df98528b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6a377bc7a58e46e28aa826fa3f084efb at {username='oryx', principal='oryx/127.15.232.253@KRBTEST.COM'} at 127.0.0.1:40814: session id = 6a377bc7a58e46e28aa826fa3f084efb-00000000000000000000000000000000
I20260810 06:45:32.985495 23242 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 06:45:32.986130 23729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 06:45:32.987187 23729 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: tablet copy: Starting download of 0 data blocks...
I20260810 06:45:32.987371 23729 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: tablet copy: Starting download of 1 WAL segments...
I20260810 06:45:32.988314 23729 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 06:45:32.989189 23242 tablet_copy_service.cc:342] P 1d4688810d5e4109b0382f56df98528b: Request end of tablet copy session 6a377bc7a58e46e28aa826fa3f084efb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.232.253@KRBTEST.COM'} at 127.0.0.1:40814
I20260810 06:45:32.989254 23242 tablet_copy_service.cc:433] P 1d4688810d5e4109b0382f56df98528b: ending tablet copy session 6a377bc7a58e46e28aa826fa3f084efb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6a377bc7a58e46e28aa826fa3f084efb
I20260810 06:45:32.991819 23729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:32.993044 23850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:32.993140 23854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:32.993243 23729 server_base.cc:1034] running on GCE node
W20260810 06:45:32.993292 23852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:32.993531 23729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:32.993667 23729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:32.994796 23729 hybrid_clock.cc:648] HybridClock initialized: now 1786344332994785 us; error 25 us; skew 500 ppm
Aug 10 06:45:32 dist-test-slave-vxj2 krb5kdc[23173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:32.996163 23729 init.cc:377] Logged in from keytab as oryx/127.15.232.253@KRBTEST.COM (short username oryx)
I20260810 06:45:32.996459 23729 webserver.cc:460] Webserver started at http://127.15.232.253:44273/ using document root <none> and password file <none>
I20260810 06:45:32.996580 23729 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:32.996636 23729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:32.997304 23729 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.997932 23861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.998059 23729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:32.998106 23729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/wal
uuid: "6a377bc7a58e46e28aa826fa3f084efb"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "ce786e86743aa0788f2eabff7eb24ae5"
server_key_iv: "6613e073b2a3ca77d040a264e0e0e866"
server_key_version: "encryptionkey@0"
I20260810 06:45:32.998165 23729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:33.002702 23729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:33.004388 23729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:33.007586 23729 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.253:34383
I20260810 06:45:33.008049 23729 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
I20260810 06:45:33.008179 23920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.253:34383 every 8 connection(s)
I20260810 06:45:33.008829 23922 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:45:33.009359 23922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: Bootstrap starting.
I20260810 06:45:33.011871 23922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:45:33.012115 23922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: Bootstrap complete.
I20260810 06:45:33.012367 23922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } }
I20260810 06:45:33.012451 23922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a377bc7a58e46e28aa826fa3f084efb, State: Initialized, Role: LEARNER
I20260810 06:45:33.012481 23922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } }
I20260810 06:45:33.012677 23922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:33.012670 23925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } }
I20260810 06:45:33.012809 23925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: This master's current role is: LEARNER
I20260810 06:45:33.014245 23939 catalog_manager.cc:1259] Loaded cluster ID: 668e327a50224196a59c433bb74e826b
I20260810 06:45:33.014285 23939 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: loading cluster ID for follower catalog manager: success
I20260810 06:45:33.015451 23939 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: acquiring CA information for follower catalog manager: success
I20260810 06:45:33.016086 23939 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:45:33.023341 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.016655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50892 (local address 127.15.232.253:34383)
0810 06:45:33.016799 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.016801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.016810 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:33.016842 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.016842 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.016851 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.016938 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.017571 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.018206 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.018937 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.019163 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.019242 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.020520 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.020522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.020522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.020530 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.022153 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.022512 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.022513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.022514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.022579 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.022786 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.022787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.022788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.022906 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.023012 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.023149 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.023234 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for oryx/127.15.232.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.252.keytab.
Entry for principal oryx/127.15.232.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.252.keytab.
I20260810 06:45:33.050637 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.252:35477
--webserver_interface=127.15.232.252
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.252.keytab
--principal=oryx/127.15.232.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:41477,127.15.232.253:34383,127.15.232.252:35477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:33.135119 23943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:33.135393 23943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:33.135417 23943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:33.136533 23943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:33.136569 23943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:33.136584 23943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:33.136596 23943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:33.136608 23943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:33.138075 23943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:41477,127.15.232.253:34383,127.15.232.252:35477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.252.keytab
--principal=oryx/127.15.232.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.252:35477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.232.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:33.138280 23943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:33.138459 23943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:45:33.143594 23943 server_base.cc:1034] running on GCE node
W20260810 06:45:33.143682 23951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:33.143827 23954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:33.143575 23948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:33.144106 23943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:33.144358 23943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:33.145491 23943 hybrid_clock.cc:648] HybridClock initialized: now 1786344333145470 us; error 37 us; skew 500 ppm
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:33.147958 23943 init.cc:377] Logged in from keytab as oryx/127.15.232.252@KRBTEST.COM (short username oryx)
I20260810 06:45:33.148343 23943 webserver.cc:460] Webserver started at http://127.15.232.252:40475/ using document root <none> and password file <none>
I20260810 06:45:33.148531 23943 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:33.148571 23943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:33.148658 23943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:33.149811 23943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data/instance:
uuid: "4eaeea758d384482a0268bfae3001d02"
format_stamp: "Formatted at 2026-08-10 06:45:33 on dist-test-slave-vxj2"
server_key: "a435a57f548143e3e39706b9bf348c34"
server_key_iv: "925a1fde681ab42b5f996897b8abfe0b"
server_key_version: "encryptionkey@0"
I20260810 06:45:33.150180 23943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/wal/instance:
uuid: "4eaeea758d384482a0268bfae3001d02"
format_stamp: "Formatted at 2026-08-10 06:45:33 on dist-test-slave-vxj2"
server_key: "a435a57f548143e3e39706b9bf348c34"
server_key_iv: "925a1fde681ab42b5f996897b8abfe0b"
server_key_version: "encryptionkey@0"
I20260810 06:45:33.151510 23943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:33.152204 23962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:33.152381 23943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:33.152491 23943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/wal
uuid: "4eaeea758d384482a0268bfae3001d02"
format_stamp: "Formatted at 2026-08-10 06:45:33 on dist-test-slave-vxj2"
server_key: "a435a57f548143e3e39706b9bf348c34"
server_key_iv: "925a1fde681ab42b5f996897b8abfe0b"
server_key_version: "encryptionkey@0"
I20260810 06:45:33.152566 23943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:33.165894 23943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:33.167806 23943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:33.167979 23943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:33.172259 23943 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.252:35477
I20260810 06:45:33.172297 24022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.252:35477 every 8 connection(s)
I20260810 06:45:33.172726 23943 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
I20260810 06:45:33.173995 24023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:33.175850 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 23943
I20260810 06:45:33.176090 24023 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.252@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.232.252@KRBTEST.COM
I20260810 06:45:33.187057 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.176953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:40820)
0810 06:45:33.177101 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.177141 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.177350 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.177558 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.177561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.177950 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.178241 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.178249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.179260 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.179261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.180350 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.180353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.180854 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.180855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.180950 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.181543 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.181555 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.183387 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.185445 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.185451 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.185454 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.185703 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.185930 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.185933 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.185933 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.186090 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.186495 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.186503 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.186679 (+ 176us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.186763 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.186893 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260810 06:45:33.187121 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.177070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.15.232.254:41477)
0810 06:45:33.177111 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.177114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.177222 (+ 108us) server_negotiation.cc:407] Connection header received
0810 06:45:33.177442 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.177444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.177455 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.177513 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.178351 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.179151 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.180446 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.180756 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.180825 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.183492 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.183494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.183494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.183502 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.185354 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.185793 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.185794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.185794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.185856 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.186191 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.186192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.186193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.186400 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.186498 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.186878 (+ 380us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.186981 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:45:33.187903 24023 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:45:33.188040 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.176501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.15.232.252:35477)
0810 06:45:33.176708 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.176717 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.176750 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:45:33.177334 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.177342 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.177371 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.177497 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.178274 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.179438 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.180445 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.180933 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.180982 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.184142 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.184149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.184149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.184156 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.185923 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.186327 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.186331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.186331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.186393 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.186684 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.186685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.186685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.187011 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.187201 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.187850 (+ 649us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.187948 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.252@KRBTEST.COM for oryx/127.15.232.253@KRBTEST.COM
I20260810 06:45:33.194628 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.188174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:34383 (local address 127.0.0.1:50902)
0810 06:45:33.188394 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.188405 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.188447 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.188617 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.188618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.188773 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.188816 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.188818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.189556 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.189557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.190056 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.190058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.190390 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.190391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.190438 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.190789 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.190793 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.191989 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.193826 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.193826 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.193827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.194002 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.194167 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.194167 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.194167 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.194217 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.194414 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.194415 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.194448 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.194498 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.194566 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260810 06:45:33.194639 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.188388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50902 (local address 127.15.232.253:34383)
0810 06:45:33.188461 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.188463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.188473 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:33.188505 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.188506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.188520 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.188587 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.188883 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.189489 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.190129 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.190324 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.190380 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.192075 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.192076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.192077 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.192083 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.193754 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.194065 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.194066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.194066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.194112 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.194255 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.194255 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.194256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.194363 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.194410 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.194516 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.194579 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:45:33.195143 24023 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.252@KRBTEST.COM for oryx/127.15.232.252@KRBTEST.COM
I20260810 06:45:33.201819 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.195499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47414 (local address 127.15.232.252:35477)
0810 06:45:33.195552 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.195554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.195566 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:45:33.195626 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.195627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.195634 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.195675 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.195970 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.196483 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.197398 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.197587 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.197636 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.199422 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.199423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.199423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.199429 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.200798 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.201113 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.201114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.201114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.201206 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.201380 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.201381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.201381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.201475 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.201526 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.201711 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.201764 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:45:33.202006 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.195471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:35477 (local address 127.0.0.1:47414)
0810 06:45:33.195525 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.195536 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.195585 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.195678 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.195678 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.195788 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.195818 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.195819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.196588 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.196589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.197308 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.197311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.197655 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.197656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.197698 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.198185 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.198190 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.199343 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.200877 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.200878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.200879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.201048 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.201268 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.201269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.201269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.201323 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.201564 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.201566 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.201609 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.201695 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.201831 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:45:33.203425 24023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: Bootstrap starting.
I20260810 06:45:33.204280 24023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:33.204568 24023 log.cc:824] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:33.205396 24023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: No bootstrap required, opened a new log
I20260810 06:45:33.206848 24023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } }
I20260810 06:45:33.206996 24023 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:33.207019 24023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eaeea758d384482a0268bfae3001d02, State: Initialized, Role: FOLLOWER
I20260810 06:45:33.207156 24023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } }
I20260810 06:45:33.207463 24029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } } }
I20260810 06:45:33.207543 24023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:33.207556 24029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:45:33.209458 24040 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:45:33.209532 24040 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:45:33.211854 23943 master_runner.cc:428] Detected that this master 4eaeea758d384482a0268bfae3001d02 is joining an existing cluster
I20260810 06:45:33.211923 23943 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.232.252:35477
I20260810 06:45:33.220886 24046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.212903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:40826)
0810 06:45:33.213100 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.213111 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.213247 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.213449 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.213450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.213679 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.213831 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.213833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.214654 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.214655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.215545 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.215548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.216046 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.216047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.216113 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.216611 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.216616 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.216956 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.219652 (+ 2696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.219653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.219653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.219886 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.220123 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.220124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.220124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.220194 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.220542 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.220543 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.220590 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.220661 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.220769 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":91,"threads_started":1}
I20260810 06:45:33.221194 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.212949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40826 (local address 127.15.232.254:41477)
0810 06:45:33.212999 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.213001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.213218 (+ 217us) server_negotiation.cc:407] Connection header received
0810 06:45:33.213339 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.213340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.213351 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.213383 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.213918 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.214567 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.215641 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.215950 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.216029 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.217049 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.217050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.217051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.217060 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.219561 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.219970 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.219971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.219972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.220048 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.220277 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.220278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.220279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.220439 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.220510 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.220706 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.221066 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 06:45:33.224359 24047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.213270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:35477 (local address 127.0.0.1:47418)
0810 06:45:33.217252 (+ 3982us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.217262 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.217556 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.217700 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.217700 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.217894 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.217964 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.217966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.218856 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.218857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.220240 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.220242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.220581 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.220582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.220640 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.221326 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.221331 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.221675 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.223345 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.223346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.223346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.223570 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.223771 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.223772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.223772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.223824 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.224120 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.224121 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.224154 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.224206 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.224286 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3966,"thread_start_us":69,"threads_started":1}
I20260810 06:45:33.224458 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.213259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47418 (local address 127.15.232.252:35477)
0810 06:45:33.213300 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.213303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.217330 (+ 4027us) server_negotiation.cc:407] Connection header received
0810 06:45:33.217628 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.217629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.217639 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.217695 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.218052 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.218766 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.220309 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.220509 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.220582 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.221764 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.221765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.221766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.221773 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.223252 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.223644 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.223645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.223646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.223704 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.223885 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.223886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.223886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.224051 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.224125 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.224229 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.224348 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":21}
I20260810 06:45:33.233088 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.213010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:34383 (local address 127.0.0.1:50914)
0810 06:45:33.213382 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.213392 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.213450 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.213588 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.213589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.213827 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.213917 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.213919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.214872 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.214873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.215730 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.215732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.216132 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.216133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.216215 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.229673 (+ 13458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.229678 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.230071 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.231933 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.231934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.231935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.232150 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.232378 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.232379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.232379 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.232445 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.232750 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.232751 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.232796 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.232869 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.232972 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":66,"threads_started":1}
I20260810 06:45:33.233108 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.212945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50914 (local address 127.15.232.253:34383)
0810 06:45:33.212991 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.212993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.213439 (+ 446us) server_negotiation.cc:407] Connection header received
0810 06:45:33.213510 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.213511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.213522 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.213583 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.213993 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.214766 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.215819 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.216067 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.216128 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.230202 (+ 14074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.230205 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.230205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.230214 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.231852 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.232231 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.232233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.232233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.232304 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.232515 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.232516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.232516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.232655 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.232720 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.232919 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.233013 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344332, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.254@KRBTEST.COM for oryx/127.15.232.252@KRBTEST.COM
I20260810 06:45:33.237692 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.230621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:35477 (local address 127.0.0.1:47420)
0810 06:45:33.230671 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.230681 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.230718 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.231000 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.231002 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.231187 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.231243 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.231246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.231841 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.231842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.232632 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.232634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.233070 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.233071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.233137 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.233552 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.233557 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.235100 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.236721 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.236722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.236723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.236892 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.237093 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.237094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.237094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.237146 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.237452 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.237452 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.237491 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.237543 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.237625 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 06:45:33.237756 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.230793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47420 (local address 127.15.232.252:35477)
0810 06:45:33.230850 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.230852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.230860 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:33.230899 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.230900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.230916 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.230976 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.231305 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.231758 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.232705 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.232956 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.233021 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.235188 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.235190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.235190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.235197 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.236635 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.236964 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.236965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.236965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.237026 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.237252 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.237253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.237253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.237389 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.237455 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.237587 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.237663 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:45:33.238400 23220 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.232.252:35477: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4eaeea758d384482a0268bfae3001d02"
member_type: NON_VOTER
last_known_addr {
host: "127.15.232.252"
port: 35477
}
attrs {
promote: true
}
}
dest_uuid: "1d4688810d5e4109b0382f56df98528b"
cas_config_opid_index: 5
I20260810 06:45:33.238682 23220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } }
I20260810 06:45:33.239046 23836 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4eaeea758d384482a0268bfae3001d02 (127.15.232.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.239440 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 23290
I20260810 06:45:33.239504 23993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:33.240197 23836 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.232.252:35477
I20260810 06:45:33.240288 23836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.240352 23836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:33.240725 23993 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 FOLLOWER]: Refusing update from remote peer 1d4688810d5e4109b0382f56df98528b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 06:45:33.241114 23836 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:33.242632 23993 raft_consensus.cc:858] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } }
W20260810 06:45:33.242714 23993 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } }
I20260810 06:45:33.243467 23993 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6a377bc7a58e46e28aa826fa3f084efb (127.15.232.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } }
I20260810 06:45:33.244987 24029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d4688810d5e4109b0382f56df98528b. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } }
I20260810 06:45:33.245062 24029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 06:45:33.245093 24029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } }
I20260810 06:45:33.245128 24029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 06:45:33.245193 24029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } }
I20260810 06:45:33.245231 24029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 06:45:33.245762 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.239157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:34383 (local address 127.0.0.1:50922)
0810 06:45:33.240993 (+ 1836us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.241007 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.241065 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.241384 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.241386 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.241387 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:33.241436 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.241438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.242639 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.242640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.244320 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.244324 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.245444 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.245445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.245522 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.245548 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.245648 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.245691 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1816}
I20260810 06:45:33.245848 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.239256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50922 (local address 127.15.232.253:34383)
0810 06:45:33.240930 (+ 1674us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.240933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.241125 (+ 192us) server_negotiation.cc:407] Connection header received
0810 06:45:33.241186 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.241188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.241226 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.241296 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:33.241523 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.242525 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.244451 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.245331 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.245439 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.245545 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.245692 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.245744 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1639}
I20260810 06:45:33.246104 23892 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 LEARNER]: Refusing update from remote peer 1d4688810d5e4109b0382f56df98528b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 06:45:33.246387 23836 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 06:45:33.247382 23892 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4eaeea758d384482a0268bfae3001d02 (127.15.232.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.248927 24056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.249022 24056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: This master's current role is: LEARNER
I20260810 06:45:33.249281 23925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d4688810d5e4109b0382f56df98528b. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: NON_VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: true } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.249395 23925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: This master's current role is: LEARNER
I20260810 06:45:33.249617 23836 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b: attempting to promote NON_VOTER 6a377bc7a58e46e28aa826fa3f084efb to VOTER
I20260810 06:45:33.249799 23836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } }
I20260810 06:45:33.250587 23892 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 LEARNER]: Refusing update from remote peer 1d4688810d5e4109b0382f56df98528b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 06:45:33.250794 23836 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 06:45:33.256035 23943 master.cc:537] Master@127.15.232.252:35477 shutting down...
I20260810 06:45:33.256421 23837 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6a377bc7a58e46e28aa826fa3f084efb (127.15.232.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.257639 23837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.257745 23837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:33.257848 23837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.257936 23837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:33.258307 23892 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6a377bc7a58e46e28aa826fa3f084efb (127.15.232.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.258759 23990 raft_consensus.cc:1467] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20260810 06:45:33.258848 23990 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 NON_PARTICIPANT]: Refusing update from remote peer 1d4688810d5e4109b0382f56df98528b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 06:45:33.259361 23925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.259469 23925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:45:33.259474 23836 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260810 06:45:33.260067 23204 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b -> Peer 4eaeea758d384482a0268bfae3001d02 (127.15.232.252:35477): Couldn't send request to peer 4eaeea758d384482a0268bfae3001d02. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 06:45:33.260761 24050 mvcc.cc:205] Tried to move back new op lower bound from 7316866384725835776 to 7316866384620826624. Current Snapshot: MvccSnapshot[applied={T|T < 7316866384633794560}]
I20260810 06:45:33.262676 23943 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260810 06:45:33.262694 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:38905
--local_ip_for_outbound_sockets=127.15.232.193
--tserver_master_addrs=127.15.232.254:41477
--webserver_port=34921
--webserver_interface=127.15.232.193
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab
--principal=oryx/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.232.254:41477,127.15.232.253:34383
--tserver_master_addrs=127.15.232.254:41477,127.15.232.253:34383,127.15.232.252:35477 with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:45:33.262797 23943 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260810 06:45:33.262844 23943 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: stopping tablet replica
I20260810 06:45:33.276913 23943 master.cc:552] Master@127.15.232.252:35477 shutdown complete.
I20260810 06:45:33.276975 23943 master_runner.cc:305] Clearing existing system tablet
I20260810 06:45:33.277113 23943 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:33.277245 23943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:33.278054 23943 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:33.278708 24063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:33.278914 23943 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:33.278983 23943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:33.279008 23943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/wal
uuid: "4eaeea758d384482a0268bfae3001d02"
format_stamp: "Formatted at 2026-08-10 06:45:33 on dist-test-slave-vxj2"
server_key: "a435a57f548143e3e39706b9bf348c34"
server_key_iv: "925a1fde681ab42b5f996897b8abfe0b"
server_key_version: "encryptionkey@0"
I20260810 06:45:33.279791 23943 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:33.281572 23943 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 06:45:33.281656 23943 log.cc:1196] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 06:45:33.281921 23943 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: Deleting consensus metadata
I20260810 06:45:33.282055 23943 master_runner.cc:315] Copying system tablet from 127.15.232.254:41477
I20260810 06:45:33.282699 23943 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: tablet copy: Beginning tablet copy session from remote peer at address 127.15.232.254:41477
I20260810 06:45:33.290225 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.283280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40834 (local address 127.15.232.254:41477)
0810 06:45:33.283327 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.283329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.283342 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:45:33.283402 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.283404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.283415 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.283497 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.283862 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.284671 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.285792 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.286085 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.286158 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.287259 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.287261 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.287261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.287269 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.288930 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.289382 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.289383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.289384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.289450 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.289667 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.289667 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.289668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.289802 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.289882 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.290054 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.290136 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:45:33.290239 24070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.283070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:40834)
0810 06:45:33.283220 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.283236 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.283337 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.283505 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.283506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.283727 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.283776 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.283778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.284757 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.284758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.285700 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.285703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.286168 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.286169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.286235 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.286771 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.286777 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.287161 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.289021 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.289022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.289023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.289285 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.289524 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.289525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.289525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.289596 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.289880 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.289881 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.289933 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.290011 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.290128 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260810 06:45:33.290850 23242 tablet_copy_service.cc:140] P 1d4688810d5e4109b0382f56df98528b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4eaeea758d384482a0268bfae3001d02 ({username='oryx', principal='oryx/127.15.232.252@KRBTEST.COM'} at 127.0.0.1:40834)
I20260810 06:45:33.290922 23242 tablet_copy_service.cc:161] P 1d4688810d5e4109b0382f56df98528b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4eaeea758d384482a0268bfae3001d02 at {username='oryx', principal='oryx/127.15.232.252@KRBTEST.COM'} at 127.0.0.1:40834: session id = 4eaeea758d384482a0268bfae3001d02-00000000000000000000000000000000
I20260810 06:45:33.291543 23242 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 06:45:33.292066 23943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 06:45:33.293361 23943 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: tablet copy: Starting download of 0 data blocks...
I20260810 06:45:33.293537 23943 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: tablet copy: Starting download of 1 WAL segments...
I20260810 06:45:33.294289 23943 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 06:45:33.295230 23242 tablet_copy_service.cc:342] P 1d4688810d5e4109b0382f56df98528b: Request end of tablet copy session 4eaeea758d384482a0268bfae3001d02-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.232.252@KRBTEST.COM'} at 127.0.0.1:40834
I20260810 06:45:33.295317 23242 tablet_copy_service.cc:433] P 1d4688810d5e4109b0382f56df98528b: ending tablet copy session 4eaeea758d384482a0268bfae3001d02-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4eaeea758d384482a0268bfae3001d02
I20260810 06:45:33.298125 23943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:33.299520 24071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:33.302241 24072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:33.302652 23943 server_base.cc:1034] running on GCE node
W20260810 06:45:33.305302 24074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:33.305495 23943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:33.305655 23943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:33.306787 23943 hybrid_clock.cc:648] HybridClock initialized: now 1786344333306777 us; error 25 us; skew 500 ppm
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:33.308532 23943 init.cc:377] Logged in from keytab as oryx/127.15.232.252@KRBTEST.COM (short username oryx)
I20260810 06:45:33.308857 23943 webserver.cc:460] Webserver started at http://127.15.232.252:44651/ using document root <none> and password file <none>
I20260810 06:45:33.308991 23943 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:33.309031 23943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:33.309916 23943 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:33.310568 24082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:33.310719 23943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:33.310781 23943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/wal
uuid: "4eaeea758d384482a0268bfae3001d02"
format_stamp: "Formatted at 2026-08-10 06:45:33 on dist-test-slave-vxj2"
server_key: "a435a57f548143e3e39706b9bf348c34"
server_key_iv: "925a1fde681ab42b5f996897b8abfe0b"
server_key_version: "encryptionkey@0"
I20260810 06:45:33.310846 23943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:33.344274 23943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:33.346321 23943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:33.349745 23943 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.252:35477
I20260810 06:45:33.349911 24141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.252:35477 every 8 connection(s)
I20260810 06:45:33.350117 23943 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
I20260810 06:45:33.351090 24143 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:45:33.351622 24143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: Bootstrap starting.
I20260810 06:45:33.354216 24143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:45:33.354489 24143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: Bootstrap complete.
I20260810 06:45:33.354713 24143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } }
I20260810 06:45:33.354808 24143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4eaeea758d384482a0268bfae3001d02, State: Initialized, Role: LEARNER
I20260810 06:45:33.354853 24143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } }
I20260810 06:45:33.355058 24143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:33.355067 24146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.355338 24146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:45:33.356711 24159 catalog_manager.cc:1259] Loaded cluster ID: 668e327a50224196a59c433bb74e826b
I20260810 06:45:33.356814 24159 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: loading cluster ID for follower catalog manager: success
I20260810 06:45:33.358103 24159 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: acquiring CA information for follower catalog manager: success
I20260810 06:45:33.358567 24159 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 06:45:33.361585 24061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:33.361794 24061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:33.361842 24061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:33.362983 24061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:33.363046 24061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:33.363097 24061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:33.364584 24061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab
--principal=oryx/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:38905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=34921
--tserver_master_addrs=127.15.232.254:41477,127.15.232.253:34383,127.15.232.252:35477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:33.364849 24061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:33.365113 24061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:33.370321 24165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:33.370327 24164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:33.370672 24061 server_base.cc:1034] running on GCE node
W20260810 06:45:33.370334 24167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:33.370958 24061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:33.371281 24061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:33.372471 24061 hybrid_clock.cc:648] HybridClock initialized: now 1786344333372451 us; error 30 us; skew 500 ppm
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:33.375574 24061 init.cc:377] Logged in from keytab as oryx/127.15.232.193@KRBTEST.COM (short username oryx)
I20260810 06:45:33.376073 24061 webserver.cc:460] Webserver started at http://127.15.232.193:34921/ using document root <none> and password file <none>
I20260810 06:45:33.376348 24061 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:33.376448 24061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:33.377924 24061 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:33.378607 24174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:33.378799 24061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:33.378911 24061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "c8fcb6da6ba24b09a602df3b23a4c521"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "9ccfeb9fef067530f9e097acbdcac39d"
server_key_iv: "65fe8cf97e28b9e018d97cb8a595eb2a"
server_key_version: "encryptionkey@0"
I20260810 06:45:33.379264 24061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:33.396399 24061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:33.399008 24061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:33.399194 24061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:33.399475 24061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:33.399809 24061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:33.399866 24061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:33.399914 24061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:33.399938 24061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:33.406212 24061 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:38905
I20260810 06:45:33.406308 24298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:38905 every 8 connection(s)
I20260810 06:45:33.406700 24061 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:33.408756 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 24061
I20260810 06:45:33.408957 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 23440
I20260810 06:45:33.426873 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:33315
--local_ip_for_outbound_sockets=127.15.232.194
--tserver_master_addrs=127.15.232.254:41477
--webserver_port=34349
--webserver_interface=127.15.232.194
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab
--principal=oryx/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.232.254:41477,127.15.232.253:34383
--tserver_master_addrs=127.15.232.254:41477,127.15.232.253:34383,127.15.232.252:35477 with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.252@KRBTEST.COM
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.253@KRBTEST.COM
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
I20260810 06:45:33.434000 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.408242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:43247 (local address 127.15.232.253:34383)
0810 06:45:33.408295 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.408297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.408306 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:33.422652 (+ 14346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.422654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.422672 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.422704 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.425055 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.425861 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.426920 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.427192 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.427227 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.430291 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.430292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.430293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.430301 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.432103 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.432728 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.432730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.432730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.432795 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.433112 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.433114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.433115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.433323 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.433372 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.433810 (+ 438us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.433904 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 06:45:33.434064 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.407594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:56191 (local address 127.15.232.254:41477)
0810 06:45:33.407649 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.407651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.407899 (+ 248us) server_negotiation.cc:407] Connection header received
0810 06:45:33.422651 (+ 14752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.422654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.422671 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.422704 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.424057 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.424925 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.427041 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.427951 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.428000 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.430347 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.430348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.430349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.430357 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.432308 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.432790 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.432792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.432792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.432859 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.433123 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.433124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.433125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.433309 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.433381 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.433828 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.433923 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:45:33.434324 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.407524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:35477 (local address 127.15.232.193:39573)
0810 06:45:33.407925 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.407937 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.422472 (+ 14535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.422906 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.422907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.424086 (+ 1179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.424140 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.424141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.425053 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.425054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.426072 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.426074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.426656 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.426657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.426806 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.427455 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.427466 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.430065 (+ 2599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.432961 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.432961 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.432961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.433198 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.433412 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.433413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.433413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.433472 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.433778 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.433779 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.433818 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.433885 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.434074 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":265,"thread_start_us":76,"threads_started":1}
I20260810 06:45:33.434327 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.408016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:34383 (local address 127.15.232.193:43247)
0810 06:45:33.408152 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.408164 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.422472 (+ 14308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.422885 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.422890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.423541 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.423917 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.423924 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.425974 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.425974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.426816 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.426817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.427327 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.427328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.427415 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.427804 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.427808 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.430176 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.432204 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.432213 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.432216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.432628 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.432877 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.432878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.432878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.433029 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.433422 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.433430 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.433623 (+ 193us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.434133 (+ 510us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.434227 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":62,"thread_start_us":96,"threads_started":1}
I20260810 06:45:33.434844 24301 heartbeater.cc:347] Connected to a master server at 127.15.232.253:34383
I20260810 06:45:33.434868 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.407533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:39573 (local address 127.15.232.252:35477)
0810 06:45:33.407683 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.407686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.408002 (+ 316us) server_negotiation.cc:407] Connection header received
0810 06:45:33.422726 (+ 14724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.422727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.422743 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.422803 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.424227 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.424912 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.426176 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.426515 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.426581 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.430198 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.430200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.430200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.430208 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.432875 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.433278 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.433280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.433280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.433342 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.433538 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.433539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.433540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.433672 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.433737 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.434651 (+ 914us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.434767 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260810 06:45:33.434960 24301 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:33.435155 24301 heartbeater.cc:511] Master 127.15.232.253:34383 requested a full tablet report, sending...
I20260810 06:45:33.435416 24304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.407524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.193:56191)
0810 06:45:33.407760 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.407796 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.422463 (+ 14667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.422885 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.422890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.423597 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.423918 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.423924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.425987 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.425988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.426914 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.426917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.428090 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.428090 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.428174 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.428643 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.428649 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.430240 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.432423 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.432424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.432424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.432694 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.432940 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.432940 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.432941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.433039 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.433423 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.433431 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.433624 (+ 193us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.435014 (+ 1390us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.435127 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":4,"thread_start_us":132,"threads_started":1}
I20260810 06:45:33.435669 24299 heartbeater.cc:347] Connected to a master server at 127.15.232.252:35477
I20260810 06:45:33.435721 24299 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:33.435740 24300 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41477
I20260810 06:45:33.435765 23881 ts_manager.cc:194] Registered new tserver with Master: c8fcb6da6ba24b09a602df3b23a4c521 (127.15.232.193:38905)
I20260810 06:45:33.435798 24299 heartbeater.cc:511] Master 127.15.232.252:35477 requested a full tablet report, sending...
I20260810 06:45:33.435806 24300 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:33.435928 24300 heartbeater.cc:511] Master 127.15.232.254:41477 requested a full tablet report, sending...
I20260810 06:45:33.436249 23220 ts_manager.cc:194] Re-registered known tserver with Master: c8fcb6da6ba24b09a602df3b23a4c521 (127.15.232.193:38905)
I20260810 06:45:33.436450 24099 ts_manager.cc:194] Registered new tserver with Master: c8fcb6da6ba24b09a602df3b23a4c521 (127.15.232.193:38905)
I20260810 06:45:33.436735 23220 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:56191
W20260810 06:45:33.516655 24307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:33.516886 24307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:33.516913 24307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:33.518515 24307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:33.518580 24307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:33.518615 24307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:33.520231 24307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab
--principal=oryx/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:33315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=34349
--tserver_master_addrs=127.15.232.254:41477,127.15.232.253:34383,127.15.232.252:35477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:33.520498 24307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:33.520763 24307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:33.526295 24313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:33.526333 24316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:33.526333 24314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:33.526579 24307 server_base.cc:1034] running on GCE node
I20260810 06:45:33.526800 24307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:33.527024 24307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:33.528189 24307 hybrid_clock.cc:648] HybridClock initialized: now 1786344333528175 us; error 27 us; skew 500 ppm
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:33.531245 24307 init.cc:377] Logged in from keytab as oryx/127.15.232.194@KRBTEST.COM (short username oryx)
I20260810 06:45:33.531721 24307 webserver.cc:460] Webserver started at http://127.15.232.194:34349/ using document root <none> and password file <none>
I20260810 06:45:33.531988 24307 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:33.532155 24307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:33.533757 24307 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:33.534631 24323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:33.534893 24307 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:33.534998 24307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "4656f0cf82604745b73b24e68adce362"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "01c4302bc12a162c6c3d32e55a6c0766"
server_key_iv: "aa3180cec757e05a34a79dc3e16efa30"
server_key_version: "encryptionkey@0"
I20260810 06:45:33.535400 24307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:33.554147 24307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:33.557129 24307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:33.557337 24307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:33.557626 24307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:33.558063 24307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:33.558120 24307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:33.558157 24307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:33.558192 24307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:33.564316 24307 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:33315
I20260810 06:45:33.564363 24449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:33315 every 8 connection(s)
I20260810 06:45:33.564738 24307 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.252@KRBTEST.COM
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.253@KRBTEST.COM
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
I20260810 06:45:33.573702 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 24307
I20260810 06:45:33.573887 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 23586
I20260810 06:45:33.578009 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.565733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:60781 (local address 127.15.232.252:35477)
0810 06:45:33.565840 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.565842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.565853 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:33.566623 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.566625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.566640 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.566689 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.567599 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.568376 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.569544 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.569779 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.569812 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.572081 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.572082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.572083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.572092 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.574195 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.576758 (+ 2563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.576759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.576760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.576831 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.577114 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.577115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.577116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.577289 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.577352 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.577846 (+ 494us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.577942 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 06:45:33.578480 24456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.565483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:35477 (local address 127.15.232.194:60781)
0810 06:45:33.565653 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.565700 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.566330 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.566738 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.566739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.567198 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.567475 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.567481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.568486 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.568487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.569448 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.569450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.569925 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.569926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.569989 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.570414 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.570419 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.571962 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.576413 (+ 4451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.576421 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.576423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.576660 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.576917 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.576918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.576918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.577034 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.577375 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.577386 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.577639 (+ 253us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.577839 (+ 200us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.578108 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":66,"thread_start_us":98,"threads_started":1}
I20260810 06:45:33.579195 24450 heartbeater.cc:347] Connected to a master server at 127.15.232.252:35477
I20260810 06:45:33.579330 24450 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:33.579586 24450 heartbeater.cc:511] Master 127.15.232.252:35477 requested a full tablet report, sending...
I20260810 06:45:33.580159 24099 ts_manager.cc:194] Registered new tserver with Master: 4656f0cf82604745b73b24e68adce362 (127.15.232.194:33315)
I20260810 06:45:33.587301 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:38029
--local_ip_for_outbound_sockets=127.15.232.195
--tserver_master_addrs=127.15.232.254:41477
--webserver_port=36451
--webserver_interface=127.15.232.195
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab
--principal=oryx/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.232.254:41477,127.15.232.253:34383
--tserver_master_addrs=127.15.232.254:41477,127.15.232.253:34383,127.15.232.252:35477 with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:45:33.587810 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.565794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:32813 (local address 127.15.232.254:41477)
0810 06:45:33.565885 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.565887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.565895 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:33.566474 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.566476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.566508 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.566559 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.567598 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.568403 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.569575 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.569884 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.569976 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.573218 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.573220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.573220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.573228 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.586225 (+ 12997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.586715 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.586716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.586717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.586793 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.587110 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.587112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.587112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.587292 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.587367 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.587632 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.587743 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 06:45:33.588189 24455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.565484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.194:32813)
0810 06:45:33.565654 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.565700 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.566319 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.566654 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.566658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.567198 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.567475 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.567481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.568487 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.568487 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.569439 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.569441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.569979 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.569980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.570077 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.570608 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.570613 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.573091 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.586329 (+ 13238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.586330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.586331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.586598 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.586938 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.586940 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.586940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.587034 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.587427 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.587428 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.587488 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.587924 (+ 436us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.588068 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":81,"thread_start_us":81,"threads_started":1}
I20260810 06:45:33.588413 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.565935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:58327 (local address 127.15.232.253:34383)
0810 06:45:33.565999 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.566001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.566009 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:33.566469 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.566470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.566484 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.566518 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.567583 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.568093 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.569125 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.569373 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.569450 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.572575 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.572577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.572577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.572587 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.586888 (+ 14301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.587368 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.587370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.587371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.587451 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.587694 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.587695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.587696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.587861 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.587912 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.588167 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.588288 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:45:33.588775 24457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.565772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:34383 (local address 127.15.232.194:58327)
0810 06:45:33.565948 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.565957 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.566329 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.566654 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.566659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.567249 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.567474 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.567479 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.568203 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.568204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.569029 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.569031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.569469 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.569470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.569626 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.570269 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.570277 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.572466 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.587004 (+ 14538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.587006 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.587006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.587264 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.587546 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.587547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.587547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.587617 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.587966 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.587967 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.588020 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.588116 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.588256 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":1,"thread_start_us":56,"threads_started":1}
I20260810 06:45:33.588858 24452 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41477
I20260810 06:45:33.588944 24452 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:33.589035 24452 heartbeater.cc:511] Master 127.15.232.254:41477 requested a full tablet report, sending...
I20260810 06:45:33.589224 24453 heartbeater.cc:347] Connected to a master server at 127.15.232.253:34383
I20260810 06:45:33.589286 24453 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:33.589372 24453 heartbeater.cc:511] Master 127.15.232.253:34383 requested a full tablet report, sending...
I20260810 06:45:33.589370 23220 ts_manager.cc:194] Re-registered known tserver with Master: 4656f0cf82604745b73b24e68adce362 (127.15.232.194:33315)
I20260810 06:45:33.589740 23881 ts_manager.cc:194] Registered new tserver with Master: 4656f0cf82604745b73b24e68adce362 (127.15.232.194:33315)
I20260810 06:45:33.589902 23220 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:32813
W20260810 06:45:33.674278 24458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:33.674508 24458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:33.674535 24458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:33.675709 24458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:33.675747 24458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:33.675774 24458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:33.677286 24458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab
--principal=oryx/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:38029
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=36451
--tserver_master_addrs=127.15.232.254:41477,127.15.232.253:34383,127.15.232.252:35477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:33.677508 24458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:33.677691 24458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:33.682927 24463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:33.683012 24466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:33.683075 24464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:33.683347 24458 server_base.cc:1034] running on GCE node
I20260810 06:45:33.683595 24458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:33.683854 24458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:33.685041 24458 hybrid_clock.cc:648] HybridClock initialized: now 1786344333685016 us; error 44 us; skew 500 ppm
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:33.687745 24458 init.cc:377] Logged in from keytab as oryx/127.15.232.195@KRBTEST.COM (short username oryx)
I20260810 06:45:33.688252 24458 webserver.cc:460] Webserver started at http://127.15.232.195:36451/ using document root <none> and password file <none>
I20260810 06:45:33.688494 24458 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:33.688581 24458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:33.690137 24458 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:33.690959 24474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:33.691167 24458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:33.691275 24458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "926b62eab226485689e4c322ef7e4a3d"
format_stamp: "Formatted at 2026-08-10 06:45:32 on dist-test-slave-vxj2"
server_key: "59dd1d8c27069b792965b6f2f9a5b24c"
server_key_iv: "1fd36795246df6c93c83973f77a76028"
server_key_version: "encryptionkey@0"
I20260810 06:45:33.691668 24458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:33.712519 24458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:33.715124 24458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:33.715252 24458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:33.715473 24458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:33.715747 24458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:33.715778 24458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:33.715801 24458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:33.715814 24458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:33.721760 24458 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:38029
I20260810 06:45:33.721849 24601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:38029 every 8 connection(s)
I20260810 06:45:33.722221 24458 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:33.724125 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 24458
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.253@KRBTEST.COM
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
Aug 10 06:45:33 dist-test-slave-vxj2 krb5kdc[23173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344333, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.252@KRBTEST.COM
I20260810 06:45:33.734748 24614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.725575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50938 (local address 127.15.232.253:34383)
0810 06:45:33.726560 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.726561 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.726568 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:45:33.726589 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.726590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.726600 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.727406 (+ 806us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.727444 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.728150 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.729256 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.729574 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.729955 (+ 381us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.731177 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.731179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.731179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.731186 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.733103 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.733542 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.733543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.733543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.733609 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.733883 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.733884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.733884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.734030 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.734459 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.734504 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.734620 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":964,"thread_start_us":55,"threads_started":1}
I20260810 06:45:33.736296 24606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.723075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.195:49491)
0810 06:45:33.723262 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.723305 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.723916 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.724241 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.724243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.724754 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.725088 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.725094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.726065 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.726067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.727320 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.727323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.727843 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.727844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.727993 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.728983 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.728988 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.731634 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.734106 (+ 2472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.734117 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.734120 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.734789 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.735009 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.735010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.735010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.735157 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.735508 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.735517 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.735700 (+ 183us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.735817 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.736016 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":50,"thread_start_us":100,"threads_started":1}
I20260810 06:45:33.736696 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.723105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:49491 (local address 127.15.232.254:41477)
0810 06:45:33.723152 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.723155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.723414 (+ 259us) server_negotiation.cc:407] Connection header received
0810 06:45:33.724069 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.724071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.724083 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.724134 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.725216 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.725974 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.727436 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.727720 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.727781 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.731742 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.731743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.731743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.731751 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.734013 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.734875 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.734877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.734877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.734945 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.735230 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.735231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.735231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.735388 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.735477 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.736514 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.736596 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:45:33.737068 24603 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41477
I20260810 06:45:33.737211 24603 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:33.737449 24603 heartbeater.cc:511] Master 127.15.232.254:41477 requested a full tablet report, sending...
I20260810 06:45:33.738222 23220 ts_manager.cc:194] Re-registered known tserver with Master: 926b62eab226485689e4c322ef7e4a3d (127.15.232.195:38029)
I20260810 06:45:33.738727 23220 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:49491
I20260810 06:45:33.739226 24608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.723398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:35477 (local address 127.15.232.195:58871)
0810 06:45:33.723528 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.723536 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.723924 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.724628 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.724629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.724833 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.725088 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.725094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.727015 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.727016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.727801 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.727803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.728269 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.728270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.728331 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.728843 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.728848 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.735550 (+ 6702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.738132 (+ 2582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.738133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.738133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.738346 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.738567 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.738567 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.738568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.738622 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.738890 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.738891 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.738950 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.739014 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.739124 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":9,"thread_start_us":73,"threads_started":1}
I20260810 06:45:33.739521 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.723526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:58871 (local address 127.15.232.252:35477)
0810 06:45:33.723603 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.723605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.723615 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:33.724016 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.724017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.724030 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.724061 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.725986 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.726724 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.727889 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.728195 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.728258 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.735656 (+ 7398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.735658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.735659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.735667 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.738044 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.738444 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.738445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.738446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.738503 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.738679 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.738680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.738680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.738796 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.738860 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.739348 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.739440 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:45:33.739864 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.723085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:43991 (local address 127.15.232.253:34383)
0810 06:45:33.723388 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.723390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.723411 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:45:33.724059 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.724061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.724075 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.724111 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.725219 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.725873 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.727416 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.727720 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.727779 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.736258 (+ 8479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.736259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.736260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.736268 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.738326 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.738807 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.738808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.738809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.738870 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.739076 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.739077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.739077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.739215 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.739277 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.739685 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.739765 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260810 06:45:33.740294 24602 heartbeater.cc:347] Connected to a master server at 127.15.232.252:35477
I20260810 06:45:33.740360 24602 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:33.740443 24602 heartbeater.cc:511] Master 127.15.232.252:35477 requested a full tablet report, sending...
I20260810 06:45:33.740470 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.723074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:34383 (local address 127.15.232.195:43991)
0810 06:45:33.723301 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.723313 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.723879 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.724229 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.724234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.724749 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:33.725088 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.725094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.725970 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.725971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.727319 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.727321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.727844 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.727845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.727993 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.728679 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:33.728688 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:33.736154 (+ 7466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:33.738466 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.738467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.738468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.738731 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.738942 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:33.738943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:33.738943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:33.739004 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:33.739308 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:33.739309 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:33.739360 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.739439 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.739555 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":49,"thread_start_us":128,"threads_started":1}
I20260810 06:45:33.740836 24099 ts_manager.cc:194] Registered new tserver with Master: 926b62eab226485689e4c322ef7e4a3d (127.15.232.195:38029)
I20260810 06:45:33.741093 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.726884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47424 (local address 127.15.232.252:35477)
0810 06:45:33.730035 (+ 3151us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.730037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.730045 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:33.730091 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.730092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.730104 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.731792 (+ 1688us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.731830 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.732579 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.733712 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.734017 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.734339 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.735534 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.735535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.735535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.735541 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.739845 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.740243 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.740244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.740244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.740305 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.740552 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.740552 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.740553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.740699 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.740788 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.740891 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.740975 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3132,"thread_start_us":63,"threads_started":1}
I20260810 06:45:33.741228 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.725683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40850 (local address 127.15.232.254:41477)
0810 06:45:33.725856 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.725858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.729782 (+ 3924us) server_negotiation.cc:407] Connection header received
0810 06:45:33.729812 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.729813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.729826 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.729909 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.731013 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.731788 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.732881 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.733265 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.733331 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.735084 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.735085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.735085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.735092 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.738685 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.739174 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.739175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.739175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.739232 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.740110 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.740111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.740112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.740248 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.740337 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.740996 (+ 659us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.741098 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260810 06:45:33.741499 24604 heartbeater.cc:347] Connected to a master server at 127.15.232.253:34383
I20260810 06:45:33.741556 24604 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:33.741592 24604 heartbeater.cc:511] Master 127.15.232.253:34383 requested a full tablet report, sending...
I20260810 06:45:33.741879 23881 ts_manager.cc:194] Registered new tserver with Master: 926b62eab226485689e4c322ef7e4a3d (127.15.232.195:38029)
I20260810 06:45:33.743288 23220 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:33.744884 23220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:33.752755 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.749045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43606 (local address 127.15.232.195:38029)
0810 06:45:33.749449 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.749454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.749466 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:45:33.749508 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.749510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.749527 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.749598 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:33.749715 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.750757 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.751875 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.752387 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.752508 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.752573 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.752632 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.752672 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":95,"threads_started":1}
I20260810 06:45:33.752740 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.748941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38029 (local address 127.0.0.1:43606)
0810 06:45:33.749255 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.749267 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.749319 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.749608 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.749609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.749611 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:33.749662 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.749664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.750854 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.750855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.751765 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.751768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.752496 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.752497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.752558 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.752561 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.752613 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.752657 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":52,"thread_start_us":62,"threads_started":1}
I20260810 06:45:33.753314 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.748763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:38905 (local address 127.0.0.1:60444)
0810 06:45:33.749239 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.749251 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.749309 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.749806 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.749806 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.749808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:33.749872 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.749874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.751097 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.751097 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.752237 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.752240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.753001 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.753002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.753069 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.753073 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.753136 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.753218 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":77,"threads_started":1}
I20260810 06:45:33.753384 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.748873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60444 (local address 127.15.232.193:38905)
0810 06:45:33.749168 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.749173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.749370 (+ 197us) server_negotiation.cc:407] Connection header received
0810 06:45:33.749455 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.749458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.749475 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.749553 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:33.749957 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.751022 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.752333 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.752917 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.752994 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.753026 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.753247 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.753299 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":146,"threads_started":1}
I20260810 06:45:33.753926 24528 tablet_service.cc:1467] Processing CreateTablet for tablet dd0824bafd754fffb561c039b32d74ac (DEFAULT_TABLE table=test-table [id=e133aede439b43b39db4350d621be7c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:33.754058 24224 tablet_service.cc:1467] Processing CreateTablet for tablet dd0824bafd754fffb561c039b32d74ac (DEFAULT_TABLE table=test-table [id=e133aede439b43b39db4350d621be7c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:33.754339 24528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd0824bafd754fffb561c039b32d74ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:33.754451 24224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd0824bafd754fffb561c039b32d74ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:33.754856 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.749085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33315 (local address 127.0.0.1:43346)
0810 06:45:33.749650 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.749659 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.749740 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.749985 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.749986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.749988 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:33.750041 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.750042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.750955 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.750956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.752393 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.752395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.754581 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.754583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.754660 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.754664 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.754725 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.754769 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":59,"threads_started":1}
I20260810 06:45:33.755168 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.749336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43346 (local address 127.15.232.194:33315)
0810 06:45:33.749516 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.749520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.749735 (+ 215us) server_negotiation.cc:407] Connection header received
0810 06:45:33.749809 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.749812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.749829 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.749917 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:33.750115 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.750867 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.753256 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.754363 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.754482 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.754541 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.754758 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.754812 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
I20260810 06:45:33.756107 24379 tablet_service.cc:1467] Processing CreateTablet for tablet dd0824bafd754fffb561c039b32d74ac (DEFAULT_TABLE table=test-table [id=e133aede439b43b39db4350d621be7c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:33.756466 24379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd0824bafd754fffb561c039b32d74ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:33.758001 24633 tablet_bootstrap.cc:492] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521: Bootstrap starting.
I20260810 06:45:33.758242 24634 tablet_bootstrap.cc:492] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d: Bootstrap starting.
I20260810 06:45:33.758700 24633 tablet_bootstrap.cc:654] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:33.758878 24634 tablet_bootstrap.cc:654] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:33.759065 24633 log.cc:824] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:33.759171 24634 log.cc:824] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:33.759752 24639 tablet_bootstrap.cc:492] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362: Bootstrap starting.
I20260810 06:45:33.760001 24633 tablet_bootstrap.cc:492] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521: No bootstrap required, opened a new log
I20260810 06:45:33.760072 24633 ts_tablet_manager.cc:1397] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:45:33.760386 24639 tablet_bootstrap.cc:654] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:33.760732 24639 log.cc:824] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:33.761732 24634 tablet_bootstrap.cc:492] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d: No bootstrap required, opened a new log
I20260810 06:45:33.761816 24634 ts_tablet_manager.cc:1397] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 06:45:33.762745 24633 raft_consensus.cc:348] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:33.762948 24633 raft_consensus.cc:374] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:33.762985 24633 raft_consensus.cc:729] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8fcb6da6ba24b09a602df3b23a4c521, State: Initialized, Role: FOLLOWER
I20260810 06:45:33.763105 24633 consensus_queue.cc:260] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:33.763406 24633 ts_tablet_manager.cc:1428] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:33.763450 24300 heartbeater.cc:503] Master 127.15.232.254:41477 was elected leader, sending a full tablet report...
I20260810 06:45:33.764448 24634 raft_consensus.cc:348] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:33.764616 24634 raft_consensus.cc:374] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:33.764664 24634 raft_consensus.cc:729] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926b62eab226485689e4c322ef7e4a3d, State: Initialized, Role: FOLLOWER
I20260810 06:45:33.764786 24634 consensus_queue.cc:260] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:33.765064 24634 ts_tablet_manager.cc:1428] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:33.765098 24603 heartbeater.cc:503] Master 127.15.232.254:41477 was elected leader, sending a full tablet report...
I20260810 06:45:33.765427 24639 tablet_bootstrap.cc:492] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362: No bootstrap required, opened a new log
I20260810 06:45:33.765722 24639 ts_tablet_manager.cc:1397] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260810 06:45:33.767959 24639 raft_consensus.cc:348] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:33.768083 24639 raft_consensus.cc:374] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:33.768105 24639 raft_consensus.cc:729] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4656f0cf82604745b73b24e68adce362, State: Initialized, Role: FOLLOWER
I20260810 06:45:33.768216 24639 consensus_queue.cc:260] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:33.768463 24639 ts_tablet_manager.cc:1428] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:33.768537 24452 heartbeater.cc:503] Master 127.15.232.254:41477 was elected leader, sending a full tablet report...
I20260810 06:45:33.802721 24641 raft_consensus.cc:482] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:33.802950 24641 raft_consensus.cc:504] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:33.803407 24641 leader_election.cc:288] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 926b62eab226485689e4c322ef7e4a3d (127.15.232.195:38029), 4656f0cf82604745b73b24e68adce362 (127.15.232.194:33315)
I20260810 06:45:33.806635 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.803801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33315 (local address 127.15.232.193:38085)
0810 06:45:33.803935 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.803946 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.804031 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.804189 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.804190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.804191 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:33.804268 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.804270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.804931 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.804932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.805785 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.805788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.806342 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.806343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.806419 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.806422 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.806515 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.806560 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":50}
I20260810 06:45:33.806631 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.803852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:38085 (local address 127.15.232.194:33315)
0810 06:45:33.803922 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.803924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.804021 (+ 97us) server_negotiation.cc:407] Connection header received
0810 06:45:33.804107 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.804107 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.804116 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.804186 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:33.804344 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.804834 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.805888 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.806258 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.806344 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.806353 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.806525 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.806565 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:45:33.806767 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.803904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:45575 (local address 127.15.232.195:38029)
0810 06:45:33.803962 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.803964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.803976 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:45:33.804012 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.804014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.804024 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.804120 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:33.804308 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.804846 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.805969 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.806360 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.806477 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.806486 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.806660 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.806699 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 06:45:33.806769 24304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.803642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38029 (local address 127.15.232.193:45575)
0810 06:45:33.803858 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.803875 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.803949 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.804128 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.804129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.804131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:33.804224 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.804226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.804939 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.804941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.805864 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.805867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.806476 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.806477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.806572 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.806575 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.806653 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.806692 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":41}
I20260810 06:45:33.807300 24550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd0824bafd754fffb561c039b32d74ac" candidate_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926b62eab226485689e4c322ef7e4a3d" is_pre_election: true
I20260810 06:45:33.807498 24550 raft_consensus.cc:2454] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8fcb6da6ba24b09a602df3b23a4c521 in term 0.
I20260810 06:45:33.807686 24400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd0824bafd754fffb561c039b32d74ac" candidate_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4656f0cf82604745b73b24e68adce362" is_pre_election: true
I20260810 06:45:33.807763 24175 leader_election.cc:302] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 926b62eab226485689e4c322ef7e4a3d, c8fcb6da6ba24b09a602df3b23a4c521; no voters:
I20260810 06:45:33.807895 24400 raft_consensus.cc:2454] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8fcb6da6ba24b09a602df3b23a4c521 in term 0.
I20260810 06:45:33.807920 24641 raft_consensus.cc:2781] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:33.807989 24641 raft_consensus.cc:482] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:33.808025 24641 raft_consensus.cc:3037] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:33.808806 24641 raft_consensus.cc:504] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:33.808956 24641 leader_election.cc:288] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [CANDIDATE]: Term 1 election: Requested vote from peers 926b62eab226485689e4c322ef7e4a3d (127.15.232.195:38029), 4656f0cf82604745b73b24e68adce362 (127.15.232.194:33315)
I20260810 06:45:33.809211 24400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd0824bafd754fffb561c039b32d74ac" candidate_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4656f0cf82604745b73b24e68adce362"
I20260810 06:45:33.809212 24550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd0824bafd754fffb561c039b32d74ac" candidate_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926b62eab226485689e4c322ef7e4a3d"
I20260810 06:45:33.809297 24400 raft_consensus.cc:3037] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:33.809299 24550 raft_consensus.cc:3037] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:33.810362 24400 raft_consensus.cc:2454] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8fcb6da6ba24b09a602df3b23a4c521 in term 1.
I20260810 06:45:33.810590 24175 leader_election.cc:302] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4656f0cf82604745b73b24e68adce362, c8fcb6da6ba24b09a602df3b23a4c521; no voters:
I20260810 06:45:33.810706 24641 raft_consensus.cc:2781] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:33.810899 24641 raft_consensus.cc:686] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 1 LEADER]: Becoming Leader. State: Replica: c8fcb6da6ba24b09a602df3b23a4c521, State: Running, Role: LEADER
I20260810 06:45:33.810905 24550 raft_consensus.cc:2454] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8fcb6da6ba24b09a602df3b23a4c521 in term 1.
I20260810 06:45:33.811007 24641 consensus_queue.cc:237] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:33.811872 23219 catalog_manager.cc:5495] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8fcb6da6ba24b09a602df3b23a4c521 (127.15.232.193). New cstate: current_term: 1 leader_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } health_report { overall_health: UNKNOWN } } }
I20260810 06:45:33.814404 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.810787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47436 (local address 127.15.232.252:35477)
0810 06:45:33.810834 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.810837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.810846 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:33.810925 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.810926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.810958 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.811038 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:33.811241 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.812026 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.813552 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.814051 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.814110 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.814184 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.814315 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.814351 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:45:33.814427 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.810684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:35477 (local address 127.0.0.1:47436)
0810 06:45:33.810780 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:33.810792 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:33.810837 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:33.811054 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:33.811056 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:33.811059 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:33.811118 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.811121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.812146 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.812147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.813469 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:33.813473 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.814133 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.814134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:33.814226 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.814231 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:45:33.814307 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 06:45:33.814369 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 06:45:33.814718 24112 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.9] Dedup: 1.7->[1.8-1.9]
I20260810 06:45:33.815135 24146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d4688810d5e4109b0382f56df98528b. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.815253 24146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:45:33.815478 23836 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b: attempting to promote NON_VOTER 4eaeea758d384482a0268bfae3001d02 to VOTER
W20260810 06:45:33.815469 24454 tablet.cc:2369] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:45:33.815510 24057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: NON_VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: true } } }
I20260810 06:45:33.815593 24057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:33.815619 23836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } }
I20260810 06:45:33.816205 23892 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 FOLLOWER]: Refusing update from remote peer 1d4688810d5e4109b0382f56df98528b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260810 06:45:33.816522 24111 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 LEARNER]: Refusing update from remote peer 1d4688810d5e4109b0382f56df98528b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260810 06:45:33.816484 24650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260810 06:45:33.816730 24650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260810 06:45:33.817348 23836 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 4eaeea758d384482a0268bfae3001d02 (127.15.232.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } } }
I20260810 06:45:33.817513 23892 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 4eaeea758d384482a0268bfae3001d02 (127.15.232.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } } }
I20260810 06:45:33.818454 23836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } } }
I20260810 06:45:33.818554 23836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:33.818506 24057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } } }
I20260810 06:45:33.818598 24057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:33.818673 24652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } } }
I20260810 06:45:33.818765 24652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:45:33.818893 24057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } } }
I20260810 06:45:33.819011 24057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:33.819093 24111 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 4eaeea758d384482a0268bfae3001d02 (127.15.232.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } } }
I20260810 06:45:33.820001 24146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d4688810d5e4109b0382f56df98528b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } } }
I20260810 06:45:33.820111 24146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:45:33.828401 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.825573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60456 (local address 127.15.232.193:38905)
0810 06:45:33.825628 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.825630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.825642 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:45:33.825712 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.825715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.825742 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.825834 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:33.826053 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.826641 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.827485 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.827704 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.827789 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.827999 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:33.828116 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:33.828229 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.828261 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.828287 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:45:33.830610 24550 raft_consensus.cc:1261] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d [term 1 FOLLOWER]: Refusing update from remote peer c8fcb6da6ba24b09a602df3b23a4c521: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:33.830610 24400 raft_consensus.cc:1261] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362 [term 1 FOLLOWER]: Refusing update from remote peer c8fcb6da6ba24b09a602df3b23a4c521: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:33.831043 24644 consensus_queue.cc:1035] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:33.831200 24641 consensus_queue.cc:1035] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [LEADER]: Connected to new peer: Peer: permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:33.834726 24654 mvcc.cc:205] Tried to move back new op lower bound from 7316866391366545408 to 7316866391290572800. Current Snapshot: MvccSnapshot[applied={T|T < 7316866391366545408}]
I20260810 06:45:33.835870 24655 mvcc.cc:205] Tried to move back new op lower bound from 7316866391366545408 to 7316866391290572800. Current Snapshot: MvccSnapshot[applied={T|T < 7316866391366545408}]
I20260810 06:45:33.837467 23219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:33.837563 23219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:33.839061 23219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d4688810d5e4109b0382f56df98528b: Sending DeleteTablet for 3 replicas of tablet dd0824bafd754fffb561c039b32d74ac
I20260810 06:45:33.839509 24379 tablet_service.cc:1514] Processing DeleteTablet for tablet dd0824bafd754fffb561c039b32d74ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:33 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:43346
I20260810 06:45:33.839851 24528 tablet_service.cc:1514] Processing DeleteTablet for tablet dd0824bafd754fffb561c039b32d74ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:33 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:43606
I20260810 06:45:33.840045 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 23186
I20260810 06:45:33.843752 24224 tablet_service.cc:1514] Processing DeleteTablet for tablet dd0824bafd754fffb561c039b32d74ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:33 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:60444
I20260810 06:45:33.844254 24664 tablet_replica.cc:330] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362: stopping tablet replica
I20260810 06:45:33.844424 24664 raft_consensus.cc:2229] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:33.844597 24664 raft_consensus.cc:2258] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:33.844838 24668 tablet_replica.cc:330] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d: stopping tablet replica
I20260810 06:45:33.845014 24669 tablet_replica.cc:330] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521: stopping tablet replica
I20260810 06:45:33.845184 24669 raft_consensus.cc:2229] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:33.845343 24668 raft_consensus.cc:2229] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:33.845520 24669 raft_consensus.cc:2258] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:33.845677 24668 raft_consensus.cc:2258] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:33.845738 24664 ts_tablet_manager.cc:1905] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:33.846477 24668 ts_tablet_manager.cc:1905] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:33.846827 24669 ts_tablet_manager.cc:1905] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:33.848165 24664 ts_tablet_manager.cc:1918] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:33.848251 24664 log.cc:1196] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/dd0824bafd754fffb561c039b32d74ac
I20260810 06:45:33.848547 24664 ts_tablet_manager.cc:1939] T dd0824bafd754fffb561c039b32d74ac P 4656f0cf82604745b73b24e68adce362: Deleting consensus metadata
W20260810 06:45:33.850122 24325 connection.cc:535] server connection from 127.0.0.1:43346 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:45:33.851071 24669 ts_tablet_manager.cc:1918] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:33.851488 24668 ts_tablet_manager.cc:1918] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:33.851651 24668 log.cc:1196] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/dd0824bafd754fffb561c039b32d74ac
I20260810 06:45:33.851776 24669 log.cc:1196] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-0/wal/wals/dd0824bafd754fffb561c039b32d74ac
I20260810 06:45:33.852140 24668 ts_tablet_manager.cc:1939] T dd0824bafd754fffb561c039b32d74ac P 926b62eab226485689e4c322ef7e4a3d: Deleting consensus metadata
I20260810 06:45:33.852634 24669 ts_tablet_manager.cc:1939] T dd0824bafd754fffb561c039b32d74ac P c8fcb6da6ba24b09a602df3b23a4c521: Deleting consensus metadata
W20260810 06:45:33.853098 24477 connection.cc:441] server connection from 127.0.0.1:43606 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43606 (request call id 1) could send its response
W20260810 06:45:33.853655 24175 connection.cc:441] server connection from 127.0.0.1:60444 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60444 (request call id 1) could send its response
I20260810 06:45:33.854313 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.839933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47442 (local address 127.15.232.252:35477)
0810 06:45:33.839972 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.839974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.840090 (+ 116us) server_negotiation.cc:407] Connection header received
0810 06:45:33.842461 (+ 2371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.842463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.842475 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.842507 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.843370 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.844158 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.846041 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.846350 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.846396 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.850910 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.850912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.850912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.850920 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.852716 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.853282 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.853283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.853283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.853344 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.853559 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.853560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.853560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.853703 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.853788 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.854131 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.854216 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:45:33.861452 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.853861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47456 (local address 127.15.232.252:35477)
0810 06:45:33.853911 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.853913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.854038 (+ 125us) server_negotiation.cc:407] Connection header received
0810 06:45:33.854143 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.854144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.854156 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.854190 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.855168 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.855897 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.856907 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.857231 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.857272 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.858836 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.858838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.858838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.858847 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.860178 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.860564 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.860565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.860566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.860627 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.860870 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.860872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.860872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.860991 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.861095 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.861251 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.861330 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 06:45:33.862535 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:33.853410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50940 (local address 127.15.232.253:34383)
0810 06:45:33.853461 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:33.853464 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:33.854011 (+ 547us) server_negotiation.cc:407] Connection header received
0810 06:45:33.854154 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:33.854155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:33.854168 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:33.854198 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:33.855440 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.856241 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.857289 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:33.857902 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:33.857972 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:33.859377 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:33.859378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:33.859379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:33.859386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:33.861380 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.861763 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.861764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.861765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.861816 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:33.862066 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:33.862067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:33.862068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:33.862182 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:33.862278 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:33.862410 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 06:45:33.862471 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:45:34.837829 24677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:34.837536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.194:34631)
0810 06:45:34.837681 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:34.837770 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
W20260810 06:45:34.838105 24452 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:41477 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
I20260810 06:45:34.838279 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:34.837993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.195:34485)
0810 06:45:34.838133 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:34.838215 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260810 06:45:34.838536 24679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:34.838287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.193:52407)
0810 06:45:34.838413 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:34.838484 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
W20260810 06:45:34.838552 24603 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:41477 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
W20260810 06:45:34.838718 24300 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:41477 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
I20260810 06:45:35.340087 24680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1d4688810d5e4109b0382f56df98528b)
I20260810 06:45:35.340150 24681 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1d4688810d5e4109b0382f56df98528b)
I20260810 06:45:35.340292 24680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } }
I20260810 06:45:35.340301 24681 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } }
I20260810 06:45:35.340619 24680 leader_election.cc:288] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d4688810d5e4109b0382f56df98528b (127.15.232.254:41477), 4eaeea758d384482a0268bfae3001d02 (127.15.232.252:35477)
I20260810 06:45:35.340878 24681 leader_election.cc:288] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d4688810d5e4109b0382f56df98528b (127.15.232.254:41477), 6a377bc7a58e46e28aa826fa3f084efb (127.15.232.253:34383)
I20260810 06:45:35.341248 24682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.340782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:53934)
0810 06:45:35.341113 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.341194 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":49,"threads_started":1}
I20260810 06:45:35.341260 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.341000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:53948)
0810 06:45:35.341101 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.341211 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":49,"threads_started":1}
W20260810 06:45:35.341382 24085 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
W20260810 06:45:35.341380 23865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
I20260810 06:45:35.341821 24682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.341734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:53966)
0810 06:45:35.341769 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.341794 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 06:45:35.341902 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.341723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:53952)
0810 06:45:35.341841 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.341872 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260810 06:45:35.341929 23865 leader_election.cc:334] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1d4688810d5e4109b0382f56df98528b (127.15.232.254:41477): Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
W20260810 06:45:35.341967 24085 leader_election.cc:334] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1d4688810d5e4109b0382f56df98528b (127.15.232.254:41477): Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
I20260810 06:45:35.344208 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.340784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:34383 (local address 127.0.0.1:56884)
0810 06:45:35.341098 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.341112 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:35.341243 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:35.341526 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:35.341527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:35.341530 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:35.341596 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.341598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.342507 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.342507 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.343313 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.343315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.343918 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.343919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.343982 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.343987 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:45:35.344069 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:45:35.344124 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":56,"threads_started":1}
I20260810 06:45:35.344202 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.340887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56884 (local address 127.15.232.253:34383)
0810 06:45:35.341012 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:35.341014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:35.341205 (+ 191us) server_negotiation.cc:407] Connection header received
0810 06:45:35.341353 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:35.341354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:35.341375 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:35.341432 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:35.341694 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.342418 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.343398 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.343832 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.343890 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.343897 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:35.344079 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 06:45:35.344140 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260810 06:45:35.344521 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.340954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.15.232.252:35477)
0810 06:45:35.341069 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:35.341071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:35.341244 (+ 173us) server_negotiation.cc:407] Connection header received
0810 06:45:35.341591 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:35.341592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:35.341603 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:35.341644 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:35.342028 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.342527 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.343575 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.344064 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.344131 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.344138 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:35.344414 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 06:45:35.344460 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":63,"threads_started":1}
I20260810 06:45:35.344481 23892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4eaeea758d384482a0268bfae3001d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6a377bc7a58e46e28aa826fa3f084efb" is_pre_election: true
I20260810 06:45:35.344630 23892 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eaeea758d384482a0268bfae3001d02 in term 1.
I20260810 06:45:35.344720 24684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.340787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:35477 (local address 127.0.0.1:58904)
0810 06:45:35.341171 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.341181 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:35.341486 (+ 305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:35.341716 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:35.341718 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:35.341720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:35.341786 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.341788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.342675 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.342676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.343482 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.343483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.344140 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.344141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.344200 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.344203 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:35.344411 (+ 208us) client_negotiation.cc:240] Negotiation successful
0810 06:45:35.344457 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":57,"threads_started":1}
I20260810 06:45:35.344878 24084 leader_election.cc:302] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4eaeea758d384482a0268bfae3001d02, 6a377bc7a58e46e28aa826fa3f084efb; no voters: 1d4688810d5e4109b0382f56df98528b
I20260810 06:45:35.345000 24681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 06:45:35.345044 24681 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1d4688810d5e4109b0382f56df98528b)
I20260810 06:45:35.345067 24681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:45:35.345103 24111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a377bc7a58e46e28aa826fa3f084efb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4eaeea758d384482a0268bfae3001d02" is_pre_election: true
I20260810 06:45:35.345816 24681 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } }
I20260810 06:45:35.345973 24681 leader_election.cc:288] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [CANDIDATE]: Term 2 election: Requested vote from peers 1d4688810d5e4109b0382f56df98528b (127.15.232.254:41477), 6a377bc7a58e46e28aa826fa3f084efb (127.15.232.253:34383)
I20260810 06:45:35.345978 24111 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a377bc7a58e46e28aa826fa3f084efb in current term 2: Already voted for candidate 4eaeea758d384482a0268bfae3001d02 in this term.
I20260810 06:45:35.346149 23892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4eaeea758d384482a0268bfae3001d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6a377bc7a58e46e28aa826fa3f084efb"
I20260810 06:45:35.346187 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.346061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:53974)
0810 06:45:35.346136 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.346163 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 06:45:35.346222 23892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:45:35.346563 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.346476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:53980)
0810 06:45:35.346521 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.346547 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 06:45:35.346689 24085 leader_election.cc:334] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1d4688810d5e4109b0382f56df98528b (127.15.232.254:41477): Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
I20260810 06:45:35.347031 23892 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eaeea758d384482a0268bfae3001d02 in term 2.
I20260810 06:45:35.347210 24084 leader_election.cc:302] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4eaeea758d384482a0268bfae3001d02, 6a377bc7a58e46e28aa826fa3f084efb; no voters: 1d4688810d5e4109b0382f56df98528b
I20260810 06:45:35.347251 23864 leader_election.cc:302] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a377bc7a58e46e28aa826fa3f084efb; no voters: 1d4688810d5e4109b0382f56df98528b, 4eaeea758d384482a0268bfae3001d02
I20260810 06:45:35.347334 24681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:45:35.347366 24680 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:45:35.347497 24681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [term 2 LEADER]: Becoming Leader. State: Replica: 4eaeea758d384482a0268bfae3001d02, State: Running, Role: LEADER
I20260810 06:45:35.347579 24681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } }
I20260810 06:45:35.347991 24691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eaeea758d384482a0268bfae3001d02. Latest consensus state: current_term: 2 leader_uuid: "4eaeea758d384482a0268bfae3001d02" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } } }
I20260810 06:45:35.348145 24691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:35.348313 24693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:35.348783 24693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:35.348886 24693 catalog_manager.cc:1259] Loaded cluster ID: 668e327a50224196a59c433bb74e826b
I20260810 06:45:35.348924 24693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:35.349119 24693 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:35.349251 24693 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: Loaded TSK: 0
I20260810 06:45:35.349377 24693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:45:35.405366 24098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:35.407263 24098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:35.408537 23892 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [term 2 FOLLOWER]: Refusing update from remote peer 4eaeea758d384482a0268bfae3001d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 06:45:35.408552 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.408418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:54020)
0810 06:45:35.408482 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.408524 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:45:35.408804 24691 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260810 06:45:35.409000 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.408845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.0.0.1:54022)
0810 06:45:35.408906 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.408958 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:45:35.409104 24085 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 -> Peer 1d4688810d5e4109b0382f56df98528b (127.15.232.254:41477): Couldn't send request to peer 1d4688810d5e4109b0382f56df98528b. Status: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 06:45:35.409631 24680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eaeea758d384482a0268bfae3001d02. Latest consensus state: current_term: 2 leader_uuid: "4eaeea758d384482a0268bfae3001d02" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } } }
I20260810 06:45:35.409752 24680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:45:35.410261 24691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "4eaeea758d384482a0268bfae3001d02" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } } }
I20260810 06:45:35.410353 24691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:35.410606 24680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "4eaeea758d384482a0268bfae3001d02" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1d4688810d5e4109b0382f56df98528b" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41477 } } peers { permanent_uuid: "6a377bc7a58e46e28aa826fa3f084efb" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 34383 } attrs { promote: false } } peers { permanent_uuid: "4eaeea758d384482a0268bfae3001d02" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 35477 } attrs { promote: false } } }
I20260810 06:45:35.410722 24680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a377bc7a58e46e28aa826fa3f084efb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:45:35.410812 24695 mvcc.cc:205] Tried to move back new op lower bound from 7316866397830569984 to 7316866397584289792. Current Snapshot: MvccSnapshot[applied={T|T < 7316866397830569984}]
I20260810 06:45:35.416069 24702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.412555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48336 (local address 127.15.232.194:33315)
0810 06:45:35.412704 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:35.412707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:35.412980 (+ 273us) server_negotiation.cc:407] Connection header received
0810 06:45:35.413011 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:35.413012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:35.413023 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:35.413254 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:35.413293 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.414000 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.415134 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.415692 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.415766 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.415775 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:35.415949 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 06:45:35.415997 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260810 06:45:35.416081 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.412449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33315 (local address 127.0.0.1:48336)
0810 06:45:35.412856 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.412865 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:35.412926 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:35.413118 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:35.413119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:35.413121 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:35.413187 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.413188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.414087 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.414088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.415040 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.415042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.415772 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.415773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.415821 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.415826 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:45:35.415924 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 06:45:35.415992 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":92,"threads_started":1}
I20260810 06:45:35.416241 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.412262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:38905 (local address 127.0.0.1:51118)
0810 06:45:35.412298 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.412312 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:35.412361 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:35.412786 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:35.412787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:35.412789 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:35.412857 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.412858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.413873 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.413874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.415223 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.415226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.415989 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.415990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.416047 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.416052 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:45:35.416109 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 06:45:35.416152 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":80}
I20260810 06:45:35.416347 24379 tablet_service.cc:1467] Processing CreateTablet for tablet a8223a7c75b64a568ff0e2d2d553c65b (DEFAULT_TABLE table=test-table [id=0250914d573a44b1a11ee654e52aad11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:35.416509 24379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8223a7c75b64a568ff0e2d2d553c65b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:35.416742 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.412323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51118 (local address 127.15.232.193:38905)
0810 06:45:35.412571 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:35.412573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:35.412581 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:35.412609 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:35.412610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:35.412621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:35.412778 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:35.412961 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.413810 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.415327 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.415888 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.415977 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.415985 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:35.416600 (+ 615us) server_negotiation.cc:299] Negotiation successful
0810 06:45:35.416669 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":65,"threads_started":1}
I20260810 06:45:35.417016 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.412362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38029 (local address 127.0.0.1:44514)
0810 06:45:35.412427 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.412436 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:35.412482 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:35.412841 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:35.412842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:35.412844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:35.412880 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.412881 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.413806 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.413807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.415223 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.415226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.416766 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.416767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.416834 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.416837 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:35.416905 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:45:35.416949 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":89}
I20260810 06:45:35.417024 24700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.412439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.15.232.195:38029)
0810 06:45:35.412682 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:35.412684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:35.412695 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:35.412726 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:35.412727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:35.412738 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:35.412952 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:35.412992 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.413729 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.416024 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.416636 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.416751 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.416761 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:35.416915 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 06:45:35.416958 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":59,"threads_started":1}
I20260810 06:45:35.418020 24703 tablet_bootstrap.cc:492] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362: Bootstrap starting.
I20260810 06:45:35.418227 24528 tablet_service.cc:1467] Processing CreateTablet for tablet a8223a7c75b64a568ff0e2d2d553c65b (DEFAULT_TABLE table=test-table [id=0250914d573a44b1a11ee654e52aad11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:35.418347 24528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8223a7c75b64a568ff0e2d2d553c65b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:35.419404 24704 tablet_bootstrap.cc:492] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d: Bootstrap starting.
I20260810 06:45:35.419382 24224 tablet_service.cc:1467] Processing CreateTablet for tablet a8223a7c75b64a568ff0e2d2d553c65b (DEFAULT_TABLE table=test-table [id=0250914d573a44b1a11ee654e52aad11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:35.419528 24224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8223a7c75b64a568ff0e2d2d553c65b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:35.419780 24704 tablet_bootstrap.cc:654] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:35.420933 24706 tablet_bootstrap.cc:492] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521: Bootstrap starting.
I20260810 06:45:35.421105 24703 tablet_bootstrap.cc:654] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:35.421386 24706 tablet_bootstrap.cc:654] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:35.421766 24703 tablet_bootstrap.cc:492] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362: No bootstrap required, opened a new log
I20260810 06:45:35.421887 24703 ts_tablet_manager.cc:1397] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 06:45:35.421984 24706 tablet_bootstrap.cc:492] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521: No bootstrap required, opened a new log
I20260810 06:45:35.422035 24706 ts_tablet_manager.cc:1397] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:35.422051 24703 raft_consensus.cc:348] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:35.422139 24703 raft_consensus.cc:374] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:35.422160 24703 raft_consensus.cc:729] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4656f0cf82604745b73b24e68adce362, State: Initialized, Role: FOLLOWER
I20260810 06:45:35.422179 24706 raft_consensus.cc:348] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:35.422217 24703 consensus_queue.cc:260] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:35.422253 24706 raft_consensus.cc:374] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:35.422277 24706 raft_consensus.cc:729] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8fcb6da6ba24b09a602df3b23a4c521, State: Initialized, Role: FOLLOWER
I20260810 06:45:35.422395 24703 ts_tablet_manager.cc:1428] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:35.422637 24704 tablet_bootstrap.cc:492] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d: No bootstrap required, opened a new log
I20260810 06:45:35.422691 24704 ts_tablet_manager.cc:1397] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 06:45:35.422865 24706 consensus_queue.cc:260] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:35.422886 24704 raft_consensus.cc:348] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:35.422956 24704 raft_consensus.cc:374] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:35.422982 24704 raft_consensus.cc:729] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926b62eab226485689e4c322ef7e4a3d, State: Initialized, Role: FOLLOWER
I20260810 06:45:35.423026 24450 heartbeater.cc:503] Master 127.15.232.252:35477 was elected leader, sending a full tablet report...
I20260810 06:45:35.423034 24704 consensus_queue.cc:260] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:35.423127 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.422576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.194:56473)
0810 06:45:35.423005 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.423080 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":71,"threads_started":1}
I20260810 06:45:35.423171 24706 ts_tablet_manager.cc:1428] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:35.423192 24704 ts_tablet_manager.cc:1428] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:35.423693 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.423462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.195:60317)
0810 06:45:35.423609 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.423653 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
I20260810 06:45:35.424222 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.423965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.194:46239)
0810 06:45:35.424139 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.424176 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":78,"threads_started":1}
I20260810 06:45:35.424454 24299 heartbeater.cc:503] Master 127.15.232.252:35477 was elected leader, sending a full tablet report...
I20260810 06:45:35.424474 24602 heartbeater.cc:503] Master 127.15.232.252:35477 was elected leader, sending a full tablet report...
I20260810 06:45:35.424615 24717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.424256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.193:60747)
0810 06:45:35.424467 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.424561 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":109,"threads_started":1}
I20260810 06:45:35.424998 24717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.424907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.193:49511)
0810 06:45:35.424943 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.424969 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 06:45:35.425199 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.425062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.195:35221)
0810 06:45:35.425108 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.425143 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260810 06:45:35.474578 24605 tablet.cc:2369] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:45:35.548381 24711 raft_consensus.cc:482] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:35.548521 24711 raft_consensus.cc:504] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:35.548998 24711 leader_election.cc:288] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8fcb6da6ba24b09a602df3b23a4c521 (127.15.232.193:38905), 926b62eab226485689e4c322ef7e4a3d (127.15.232.195:38029)
I20260810 06:45:35.552474 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.549303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:38905 (local address 127.15.232.194:56819)
0810 06:45:35.549374 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.549390 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:35.549481 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:35.549629 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:35.549630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:35.549634 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:35.549813 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.549816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.550494 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.550496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.551412 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.551415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.552154 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.552156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.552258 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.552264 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:45:35.552322 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 06:45:35.552364 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:45:35.552495 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.549434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:56819 (local address 127.15.232.193:38905)
0810 06:45:35.549485 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:35.549488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:35.549516 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:45:35.549551 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:35.549552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:35.549562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:35.549624 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:35.549904 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.550398 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.551504 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.552073 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.552153 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.552165 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:35.552337 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 06:45:35.552381 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 06:45:35.552611 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.549426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38029 (local address 127.15.232.194:58615)
0810 06:45:35.549478 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.549489 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:35.549542 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:35.549770 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:35.549771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:35.549773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:35.549836 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.549838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.550618 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.550619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.551518 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:35.551521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.552353 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.552355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:35.552424 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.552428 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:35.552485 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 06:45:35.552533 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 06:45:35.552615 24700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.549529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:58615 (local address 127.15.232.195:38029)
0810 06:45:35.549578 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:35.549580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:35.549590 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:35.549626 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:35.549627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:35.549638 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:35.549709 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:35.549956 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.550528 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.551624 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.552215 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.552317 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.552328 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:35.552492 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 06:45:35.552538 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:45:35.552762 24247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8223a7c75b64a568ff0e2d2d553c65b" candidate_uuid: "4656f0cf82604745b73b24e68adce362" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" is_pre_election: true
I20260810 06:45:35.552902 24247 raft_consensus.cc:2454] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4656f0cf82604745b73b24e68adce362 in term 0.
I20260810 06:45:35.552909 24550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8223a7c75b64a568ff0e2d2d553c65b" candidate_uuid: "4656f0cf82604745b73b24e68adce362" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926b62eab226485689e4c322ef7e4a3d" is_pre_election: true
I20260810 06:45:35.553030 24550 raft_consensus.cc:2454] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4656f0cf82604745b73b24e68adce362 in term 0.
I20260810 06:45:35.553122 24324 leader_election.cc:302] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4656f0cf82604745b73b24e68adce362, c8fcb6da6ba24b09a602df3b23a4c521; no voters:
I20260810 06:45:35.553282 24711 raft_consensus.cc:2781] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:35.553330 24711 raft_consensus.cc:482] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:35.553355 24711 raft_consensus.cc:3037] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:35.554103 24711 raft_consensus.cc:504] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:35.554217 24711 leader_election.cc:288] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [CANDIDATE]: Term 1 election: Requested vote from peers c8fcb6da6ba24b09a602df3b23a4c521 (127.15.232.193:38905), 926b62eab226485689e4c322ef7e4a3d (127.15.232.195:38029)
I20260810 06:45:35.554514 24550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8223a7c75b64a568ff0e2d2d553c65b" candidate_uuid: "4656f0cf82604745b73b24e68adce362" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926b62eab226485689e4c322ef7e4a3d"
I20260810 06:45:35.554530 24247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8223a7c75b64a568ff0e2d2d553c65b" candidate_uuid: "4656f0cf82604745b73b24e68adce362" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8fcb6da6ba24b09a602df3b23a4c521"
I20260810 06:45:35.554623 24247 raft_consensus.cc:3037] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:35.554628 24550 raft_consensus.cc:3037] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:35.555369 24247 raft_consensus.cc:2454] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4656f0cf82604745b73b24e68adce362 in term 1.
I20260810 06:45:35.555563 24324 leader_election.cc:302] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4656f0cf82604745b73b24e68adce362, c8fcb6da6ba24b09a602df3b23a4c521; no voters:
I20260810 06:45:35.555665 24711 raft_consensus.cc:2781] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:35.555632 24550 raft_consensus.cc:2454] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4656f0cf82604745b73b24e68adce362 in term 1.
I20260810 06:45:35.555832 24711 raft_consensus.cc:686] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 1 LEADER]: Becoming Leader. State: Replica: 4656f0cf82604745b73b24e68adce362, State: Running, Role: LEADER
I20260810 06:45:35.555935 24711 consensus_queue.cc:237] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } }
I20260810 06:45:35.556221 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.556086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.194:49621)
0810 06:45:35.556147 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.556189 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:45:35.556828 24099 catalog_manager.cc:5495] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4656f0cf82604745b73b24e68adce362 (127.15.232.194). New cstate: current_term: 1 leader_uuid: "4656f0cf82604745b73b24e68adce362" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4656f0cf82604745b73b24e68adce362" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33315 } health_report { overall_health: HEALTHY } } }
I20260810 06:45:35.599323 24702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.596498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48350 (local address 127.15.232.194:33315)
0810 06:45:35.596571 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:35.596573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:35.596583 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:35.596645 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:35.596646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:35.596667 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:35.596748 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:35.596897 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.597423 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.598367 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:35.598593 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:35.598698 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:35.598920 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:35.599030 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:35.599101 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:35.599142 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 06:45:35.599166 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:45:35.600696 24550 raft_consensus.cc:1261] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d [term 1 FOLLOWER]: Refusing update from remote peer 4656f0cf82604745b73b24e68adce362: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:35.600776 24247 raft_consensus.cc:1261] T a8223a7c75b64a568ff0e2d2d553c65b P c8fcb6da6ba24b09a602df3b23a4c521 [term 1 FOLLOWER]: Refusing update from remote peer 4656f0cf82604745b73b24e68adce362: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:35.601115 24711 consensus_queue.cc:1035] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [LEADER]: Connected to new peer: Peer: permanent_uuid: "926b62eab226485689e4c322ef7e4a3d" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:35.601311 24719 consensus_queue.cc:1035] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8fcb6da6ba24b09a602df3b23a4c521" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:35.602438 24717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.602317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.193:50781)
0810 06:45:35.602367 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.602402 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 06:45:35.602711 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.602605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.195:40883)
0810 06:45:35.602648 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.602679 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 06:45:35.603960 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.603788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.195:49409)
0810 06:45:35.603900 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.603929 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 06:45:35.604031 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.603928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.194:33053)
0810 06:45:35.603973 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.604002 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 06:45:35.604353 24717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:35.604258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41477 (local address 127.15.232.193:34983)
0810 06:45:35.604294 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.604318 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 06:45:35.604597 24723 mvcc.cc:205] Tried to move back new op lower bound from 7316866398617395200 to 7316866398437691392. Current Snapshot: MvccSnapshot[applied={T|T < 7316866398617395200}]
I20260810 06:45:35.605970 24099 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:35.606060 24099 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:35.607329 24099 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4eaeea758d384482a0268bfae3001d02: Sending DeleteTablet for 3 replicas of tablet a8223a7c75b64a568ff0e2d2d553c65b
I20260810 06:45:35.607614 24224 tablet_service.cc:1514] Processing DeleteTablet for tablet a8223a7c75b64a568ff0e2d2d553c65b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:35 UTC) from {username='oryx', principal='oryx/127.15.232.252@KRBTEST.COM'} at 127.0.0.1:51118
I20260810 06:45:35.607677 24528 tablet_service.cc:1514] Processing DeleteTablet for tablet a8223a7c75b64a568ff0e2d2d553c65b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:35 UTC) from {username='oryx', principal='oryx/127.15.232.252@KRBTEST.COM'} at 127.0.0.1:44514
I20260810 06:45:35.608033 24379 tablet_service.cc:1514] Processing DeleteTablet for tablet a8223a7c75b64a568ff0e2d2d553c65b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:35 UTC) from {username='oryx', principal='oryx/127.15.232.252@KRBTEST.COM'} at 127.0.0.1:48336
I20260810 06:45:35.608328 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 24061
I20260810 06:45:35.614828 24734 tablet_replica.cc:330] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362: stopping tablet replica
I20260810 06:45:35.614918 24734 raft_consensus.cc:2229] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:35.615042 24734 raft_consensus.cc:2258] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:35.615265 24733 tablet_replica.cc:330] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d: stopping tablet replica
I20260810 06:45:35.615358 24733 raft_consensus.cc:2229] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:35.615417 24733 raft_consensus.cc:2258] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:35.615598 24734 ts_tablet_manager.cc:1905] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:35.615734 24733 ts_tablet_manager.cc:1905] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:35.617311 24733 ts_tablet_manager.cc:1918] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:35.617390 24733 log.cc:1196] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/a8223a7c75b64a568ff0e2d2d553c65b
I20260810 06:45:35.617636 24733 ts_tablet_manager.cc:1939] T a8223a7c75b64a568ff0e2d2d553c65b P 926b62eab226485689e4c322ef7e4a3d: Deleting consensus metadata
I20260810 06:45:35.617801 24734 ts_tablet_manager.cc:1918] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:35.617869 24734 log.cc:1196] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/a8223a7c75b64a568ff0e2d2d553c65b
I20260810 06:45:35.618021 24083 catalog_manager.cc:4841] TS 926b62eab226485689e4c322ef7e4a3d (127.15.232.195:38029): tablet a8223a7c75b64a568ff0e2d2d553c65b (table test-table [id=0250914d573a44b1a11ee654e52aad11]) successfully deleted
I20260810 06:45:35.618105 24734 ts_tablet_manager.cc:1939] T a8223a7c75b64a568ff0e2d2d553c65b P 4656f0cf82604745b73b24e68adce362: Deleting consensus metadata
I20260810 06:45:35.618606 24083 catalog_manager.cc:4841] TS 4656f0cf82604745b73b24e68adce362 (127.15.232.194:33315): tablet a8223a7c75b64a568ff0e2d2d553c65b (table test-table [id=0250914d573a44b1a11ee654e52aad11]) successfully deleted
W20260810 06:45:35.629557 24688 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:35.629276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:38905 (local address 127.0.0.1:51130)
0810 06:45:35.629347 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:35.629360 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:35.629410 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:35.629479 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:38905: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:45:35.629820 24083 catalog_manager.cc:4584] TS c8fcb6da6ba24b09a602df3b23a4c521 (127.15.232.193:38905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8223a7c75b64a568ff0e2d2d553c65b: Network error: Client connection negotiation failed: client connection to 127.15.232.193:38905: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260810 06:45:35.634361 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 24307
I20260810 06:45:35.645903 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 24458
I20260810 06:45:35.657691 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 23729
I20260810 06:45:35.666678 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 23943
2026-08-10T06:45:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4934 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:35 dist-test-slave-vxj2 krb5kdc[24737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:35 dist-test-slave-vxj2 krb5kdc[24737](info): set up 2 sockets
Aug 10 06:45:35 dist-test-slave-vxj2 krb5kdc[24737](info): commencing operation
krb5kdc: starting...
W20260810 06:45:37.212196 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.505s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:37 dist-test-slave-vxj2 krb5kdc[24737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.254.keytab.
Entry for principal oryx/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.254.keytab.
I20260810 06:45:37.339672 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:35699
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.254.keytab
--principal=oryx/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:35699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:37.427954 24753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:37.428175 24753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:37.428197 24753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:37.429298 24753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:37.429335 24753 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:45:37.429348 24753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:37.429361 24753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:37.429373 24753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:37.429385 24753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:37.430847 24753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:35699
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.254.keytab
--principal=oryx/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:35699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:37.431043 24753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:37.431219 24753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:37.436107 24759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:37.436127 24760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:37.436246 24753 server_base.cc:1034] running on GCE node
W20260810 06:45:37.436154 24762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:37.436661 24753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:37.436946 24753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:37.438099 24753 hybrid_clock.cc:648] HybridClock initialized: now 1786344337438060 us; error 56 us; skew 500 ppm
Aug 10 06:45:37 dist-test-slave-vxj2 krb5kdc[24737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344337, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:37.440675 24753 init.cc:377] Logged in from keytab as oryx/127.15.232.254@KRBTEST.COM (short username oryx)
I20260810 06:45:37.441254 24753 webserver.cc:460] Webserver started at http://127.15.232.254:40815/ using document root <none> and password file <none>
I20260810 06:45:37.441504 24753 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:37.441550 24753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:37.441720 24753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:37.442847 24753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "7553efd276804d57a18561dadc7cbba9"
format_stamp: "Formatted at 2026-08-10 06:45:37 on dist-test-slave-vxj2"
server_key: "4e81955f29097c866a9fbd12a8c464d2"
server_key_iv: "fb54f1c8b4d1fc4bfbb80c17585949db"
server_key_version: "encryptionkey@0"
I20260810 06:45:37.443205 24753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "7553efd276804d57a18561dadc7cbba9"
format_stamp: "Formatted at 2026-08-10 06:45:37 on dist-test-slave-vxj2"
server_key: "4e81955f29097c866a9fbd12a8c464d2"
server_key_iv: "fb54f1c8b4d1fc4bfbb80c17585949db"
server_key_version: "encryptionkey@0"
I20260810 06:45:37.444574 24753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:37.445384 24770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:37.445595 24753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:37.445685 24753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "7553efd276804d57a18561dadc7cbba9"
format_stamp: "Formatted at 2026-08-10 06:45:37 on dist-test-slave-vxj2"
server_key: "4e81955f29097c866a9fbd12a8c464d2"
server_key_iv: "fb54f1c8b4d1fc4bfbb80c17585949db"
server_key_version: "encryptionkey@0"
I20260810 06:45:37.445741 24753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:37.463384 24753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:37.465771 24753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:37.465893 24753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:37.470494 24753 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:35699
I20260810 06:45:37.470606 24838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:35699 every 8 connection(s)
I20260810 06:45:37.470922 24753 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:37.471634 24839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:37.474059 24839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9: Bootstrap starting.
I20260810 06:45:37.474633 24839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:37.474838 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 24753
I20260810 06:45:37.474941 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:37.474987 24839 log.cc:824] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:37.475189 16291 external_mini_cluster.cc:1428] Setting key 64abbf75032356ac40b5973882ee4ef8
I20260810 06:45:37.475859 24839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9: No bootstrap required, opened a new log
I20260810 06:45:37.478399 24839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7553efd276804d57a18561dadc7cbba9" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35699 } }
I20260810 06:45:37.478570 24839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:37.478621 24839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7553efd276804d57a18561dadc7cbba9, State: Initialized, Role: FOLLOWER
I20260810 06:45:37.478773 24839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7553efd276804d57a18561dadc7cbba9" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35699 } }
I20260810 06:45:37.478873 24839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:37.478919 24839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:37.478968 24839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:37.479517 24839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7553efd276804d57a18561dadc7cbba9" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35699 } }
I20260810 06:45:37.479661 24839 leader_election.cc:302] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7553efd276804d57a18561dadc7cbba9; no voters:
I20260810 06:45:37.479876 24839 leader_election.cc:288] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:37.479902 24844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:37.480010 24844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [term 1 LEADER]: Becoming Leader. State: Replica: 7553efd276804d57a18561dadc7cbba9, State: Running, Role: LEADER
I20260810 06:45:37.480108 24844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7553efd276804d57a18561dadc7cbba9" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35699 } }
I20260810 06:45:37.480152 24839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:37.480468 24845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7553efd276804d57a18561dadc7cbba9. Latest consensus state: current_term: 1 leader_uuid: "7553efd276804d57a18561dadc7cbba9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7553efd276804d57a18561dadc7cbba9" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35699 } } }
I20260810 06:45:37.480474 24846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7553efd276804d57a18561dadc7cbba9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7553efd276804d57a18561dadc7cbba9" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35699 } } }
I20260810 06:45:37.480683 24845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:37.480701 24846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:37.481120 24853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:37.481675 24853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:45:37 dist-test-slave-vxj2 krb5kdc[24737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
I20260810 06:45:37.483431 24853 catalog_manager.cc:1347] Generated new cluster ID: 77cf0ab9820245ecb82b09da56af3f63
I20260810 06:45:37.483484 24853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:37.486765 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:37.475736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40586 (local address 127.15.232.254:35699)
0810 06:45:37.475972 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:37.475982 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:37.476015 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:45:37.476443 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:37.476450 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:37.476496 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:37.476652 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:37.477351 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.478249 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.479195 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.479657 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.479784 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:37.483052 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:37.483056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:37.483056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:37.483069 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:37.484921 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:37.485370 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:37.485373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:37.485374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:37.485468 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:37.485689 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:37.485690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:37.485690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:37.485928 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:37.486107 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:37.486194 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 06:45:37.486352 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":134,"threads_started":1}
I20260810 06:45:37.506533 24853 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:37.507138 24853 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:37.513692 24853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9: Generated new TSK 0
I20260810 06:45:37.513908 24853 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab.
Entry for principal oryx/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab.
I20260810 06:45:37.552618 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35699
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab
--principal=oryx/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:37.649863 24867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:37.650090 24867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:37.650130 24867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:37.651284 24867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:37.651352 24867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:37.651388 24867 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:45:37.651448 24867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:37.652987 24867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab
--principal=oryx/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.254:35699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:37.653278 24867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:37.653515 24867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:37.658830 24872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:37.658993 24867 server_base.cc:1034] running on GCE node
W20260810 06:45:37.658830 24873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:37.658890 24875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:37.659301 24867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:37.659507 24867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:37.660665 24867 hybrid_clock.cc:648] HybridClock initialized: now 1786344337660648 us; error 31 us; skew 500 ppm
Aug 10 06:45:37 dist-test-slave-vxj2 krb5kdc[24737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344337, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:37.663565 24867 init.cc:377] Logged in from keytab as oryx/127.15.232.193@KRBTEST.COM (short username oryx)
I20260810 06:45:37.664036 24867 webserver.cc:460] Webserver started at http://127.15.232.193:41797/ using document root <none> and password file <none>
I20260810 06:45:37.664289 24867 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:37.664358 24867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:37.664520 24867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:37.665680 24867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "4db1bb333a7141ee9e342a77941df9f5"
format_stamp: "Formatted at 2026-08-10 06:45:37 on dist-test-slave-vxj2"
server_key: "991a0aa38f398ca29ef80731410ce64e"
server_key_iv: "365dae830974b6a30df174f3bc187bef"
server_key_version: "encryptionkey@0"
I20260810 06:45:37.666062 24867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "4db1bb333a7141ee9e342a77941df9f5"
format_stamp: "Formatted at 2026-08-10 06:45:37 on dist-test-slave-vxj2"
server_key: "991a0aa38f398ca29ef80731410ce64e"
server_key_iv: "365dae830974b6a30df174f3bc187bef"
server_key_version: "encryptionkey@0"
I20260810 06:45:37.667503 24867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:37.668349 24882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:37.668551 24867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:37.668653 24867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "4db1bb333a7141ee9e342a77941df9f5"
format_stamp: "Formatted at 2026-08-10 06:45:37 on dist-test-slave-vxj2"
server_key: "991a0aa38f398ca29ef80731410ce64e"
server_key_iv: "365dae830974b6a30df174f3bc187bef"
server_key_version: "encryptionkey@0"
I20260810 06:45:37.668733 24867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:37.697011 24867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:37.699891 24867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:37.700078 24867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:37.700657 24867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:37.700736 24867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:37.700778 24867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:37.700826 24867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:45:37 dist-test-slave-vxj2 krb5kdc[24737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344337, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
I20260810 06:45:37.708948 24867 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:42115
I20260810 06:45:37.709411 24867 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:37.709439 25008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:42115 every 8 connection(s)
I20260810 06:45:37.711704 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:37.701255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:43577 (local address 127.15.232.254:35699)
0810 06:45:37.701311 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:37.701313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:37.701674 (+ 361us) server_negotiation.cc:407] Connection header received
0810 06:45:37.702203 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:37.702205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:37.702218 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:37.702271 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:37.703223 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.704093 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.705228 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.705462 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.705548 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:37.708407 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:37.708410 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:37.708410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:37.708421 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:37.710195 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:37.710680 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:37.710682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:37.710683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:37.710760 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:37.711021 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:37.711022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:37.711022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:37.711159 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:37.711234 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:37.711518 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 06:45:37.711612 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 06:45:37.711884 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:37.701249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35699 (local address 127.15.232.193:43577)
0810 06:45:37.701440 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:37.701478 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:37.702084 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:37.702378 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:37.702382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:37.702807 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:37.703128 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:37.703136 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.704215 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.704216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:37.705090 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:37.705093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.705571 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.705572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:37.705723 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:37.706293 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:37.706303 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:37.708295 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:37.710307 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:37.710315 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:37.710318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:37.710583 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:37.710834 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:37.710835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:37.710836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:37.710947 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:37.711243 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:37.711250 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:37.711385 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 06:45:37.711496 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 06:45:37.711655 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
I20260810 06:45:37.712407 25009 heartbeater.cc:347] Connected to a master server at 127.15.232.254:35699
I20260810 06:45:37.712496 25009 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:37.712694 25009 heartbeater.cc:511] Master 127.15.232.254:35699 requested a full tablet report, sending...
I20260810 06:45:37.713344 24787 ts_manager.cc:194] Registered new tserver with Master: 4db1bb333a7141ee9e342a77941df9f5 (127.15.232.193:42115)
I20260810 06:45:37.714016 24787 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:43577
I20260810 06:45:37.719076 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 24867
I20260810 06:45:37.719153 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:37.713530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:45891 (local address 127.15.232.254:35699)
0810 06:45:37.713576 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:37.713578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:37.713705 (+ 127us) server_negotiation.cc:407] Connection header received
0810 06:45:37.713777 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:37.713778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:37.713788 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:37.713836 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:37.714143 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.714725 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.715447 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.715654 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.715746 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:37.716676 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:37.716678 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:37.716678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:37.716685 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:37.717997 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:37.718347 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:37.718348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:37.718349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:37.718409 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:37.718613 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:37.718614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:37.718614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:37.718729 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:37.718809 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:37.718995 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 06:45:37.719066 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:45:37.719172 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:37.713480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35699 (local address 127.15.232.193:45891)
0810 06:45:37.713632 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:37.713649 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:37.713720 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:37.713838 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:37.713839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:37.714013 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:37.714067 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:37.714068 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.714830 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.714831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:37.715363 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:37.715365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.715742 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.715743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:37.715814 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:37.716226 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:37.716230 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:37.716550 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:37.718092 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:37.718093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:37.718094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:37.718274 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:37.718478 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:37.718479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:37.718479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:37.718546 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:37.718811 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:37.718812 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:37.718870 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 06:45:37.718949 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:45:37.719061 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
I20260810 06:45:37.719211 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:37.719457 16291 external_mini_cluster.cc:1428] Setting key b3302089a513a688b4d22d1b6b26cc64
WARNING: no policy specified for oryx/127.15.232.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab.
Entry for principal oryx/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab.
I20260810 06:45:37.746328 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35699
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab
--principal=oryx/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:37.829869 25018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:37.830077 25018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:37.830101 25018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:37.831346 25018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:37.831389 25018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:37.831410 25018 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:45:37.831427 25018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:37.832887 25018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab
--principal=oryx/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.254:35699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:37.833077 25018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:37.833324 25018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:37.838464 25026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:37.838471 25024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:37.838469 25023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:37.838754 25018 server_base.cc:1034] running on GCE node
I20260810 06:45:37.838989 25018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:37.839226 25018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:37.840369 25018 hybrid_clock.cc:648] HybridClock initialized: now 1786344337840358 us; error 30 us; skew 500 ppm
Aug 10 06:45:37 dist-test-slave-vxj2 krb5kdc[24737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344337, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:37.842980 25018 init.cc:377] Logged in from keytab as oryx/127.15.232.194@KRBTEST.COM (short username oryx)
I20260810 06:45:37.843509 25018 webserver.cc:460] Webserver started at http://127.15.232.194:34407/ using document root <none> and password file <none>
I20260810 06:45:37.843748 25018 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:37.843878 25018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:37.844039 25018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:37.845129 25018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "6e68488536e348e88ee3491a468288fe"
format_stamp: "Formatted at 2026-08-10 06:45:37 on dist-test-slave-vxj2"
server_key: "3d5d7cbb6c3157ce874299e605ef5870"
server_key_iv: "52af48a5d6b68eff699e9758bba3ad64"
server_key_version: "encryptionkey@0"
I20260810 06:45:37.845575 25018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "6e68488536e348e88ee3491a468288fe"
format_stamp: "Formatted at 2026-08-10 06:45:37 on dist-test-slave-vxj2"
server_key: "3d5d7cbb6c3157ce874299e605ef5870"
server_key_iv: "52af48a5d6b68eff699e9758bba3ad64"
server_key_version: "encryptionkey@0"
I20260810 06:45:37.846976 25018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:37.847687 25034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:37.847930 25018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:37.848057 25018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "6e68488536e348e88ee3491a468288fe"
format_stamp: "Formatted at 2026-08-10 06:45:37 on dist-test-slave-vxj2"
server_key: "3d5d7cbb6c3157ce874299e605ef5870"
server_key_iv: "52af48a5d6b68eff699e9758bba3ad64"
server_key_version: "encryptionkey@0"
I20260810 06:45:37.848125 25018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:37.873924 25018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:37.876016 25018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:37.876195 25018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:37.876822 25018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:37.876864 25018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:37.876888 25018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:37.876900 25018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:45:37 dist-test-slave-vxj2 krb5kdc[24737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344337, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
I20260810 06:45:37.884718 25018 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:39153
I20260810 06:45:37.884778 25165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:39153 every 8 connection(s)
I20260810 06:45:37.885214 25018 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:37.887212 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:37.877300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:36811 (local address 127.15.232.254:35699)
0810 06:45:37.877361 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:37.877363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:37.877576 (+ 213us) server_negotiation.cc:407] Connection header received
0810 06:45:37.878107 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:37.878108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:37.878122 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:37.878208 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:37.878943 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.879842 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.881139 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.881398 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.881485 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:37.884427 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:37.884429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:37.884429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:37.884437 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:37.886078 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:37.886468 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:37.886469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:37.886470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:37.886521 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:37.886701 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:37.886702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:37.886702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:37.886803 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:37.886855 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:37.887085 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 06:45:37.887148 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 06:45:37.887475 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:37.877275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35699 (local address 127.15.232.194:36811)
0810 06:45:37.877433 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:37.877476 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:37.878002 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:37.878322 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:37.878326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:37.878623 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:37.878815 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:37.878820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.879953 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.879954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:37.880988 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:37.881010 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.881502 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.881504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:37.881632 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:37.882239 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:37.882268 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:37.884312 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:37.886157 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:37.886161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:37.886163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:37.886392 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:37.886562 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:37.886563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:37.886563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:37.886650 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:37.886865 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:37.886871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:37.886987 (+ 116us) client_negotiation.cc:769] Sending connection context
0810 06:45:37.887074 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:45:37.887197 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":111,"threads_started":1}
I20260810 06:45:37.888005 25166 heartbeater.cc:347] Connected to a master server at 127.15.232.254:35699
I20260810 06:45:37.888087 25166 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:37.888283 25166 heartbeater.cc:511] Master 127.15.232.254:35699 requested a full tablet report, sending...
I20260810 06:45:37.888731 24787 ts_manager.cc:194] Registered new tserver with Master: 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153)
I20260810 06:45:37.889266 24787 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:36811
I20260810 06:45:37.892072 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 25018
I20260810 06:45:37.892161 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:37.892386 16291 external_mini_cluster.cc:1428] Setting key 17775691461b7de4ad68b3cc2fc5725a
I20260810 06:45:37.895005 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:37.889190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:52403 (local address 127.15.232.254:35699)
0810 06:45:37.889229 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:37.889231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:37.889284 (+ 53us) server_negotiation.cc:407] Connection header received
0810 06:45:37.889354 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:37.889355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:37.889366 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:37.889423 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:37.889805 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.890256 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.890928 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.891102 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.891150 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:37.891959 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:37.891961 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:37.891961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:37.891967 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:37.893811 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:37.894141 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:37.894142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:37.894143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:37.894204 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:37.894403 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:37.894404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:37.894405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:37.894604 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:37.894680 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:37.894873 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 06:45:37.894947 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:45:37.895087 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:37.889060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35699 (local address 127.15.232.194:52403)
0810 06:45:37.889219 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:37.889231 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:37.889294 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:37.889430 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:37.889431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:37.889645 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:37.889723 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:37.889725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.890350 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.890351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:37.890850 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:37.890851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:37.891164 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:37.891164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:37.891212 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:37.891545 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:37.891550 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:37.891814 (+ 264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:37.893898 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:37.893899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:37.893900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:37.894067 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:37.894295 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:37.894296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:37.894296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:37.894344 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:37.894684 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:37.894685 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:37.894740 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 06:45:37.894826 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:45:37.894946 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for oryx/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab.
Entry for principal oryx/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab.
I20260810 06:45:37.920610 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35699
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab
--principal=oryx/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:38.004014 25177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:38.004262 25177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:38.004305 25177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:38.005566 25177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:38.005631 25177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:38.005664 25177 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:45:38.005702 25177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:38.007308 25177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab
--principal=oryx/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.254:35699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:38.007591 25177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:38.007843 25177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:38.013188 25184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:38.013190 25187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:38.013440 25177 server_base.cc:1034] running on GCE node
W20260810 06:45:38.013221 25183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:38.013782 25177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:38.014107 25177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:38.015285 25177 hybrid_clock.cc:648] HybridClock initialized: now 1786344338015260 us; error 37 us; skew 500 ppm
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[24737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344338, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:38.018033 25177 init.cc:377] Logged in from keytab as oryx/127.15.232.195@KRBTEST.COM (short username oryx)
I20260810 06:45:38.018536 25177 webserver.cc:460] Webserver started at http://127.15.232.195:38941/ using document root <none> and password file <none>
I20260810 06:45:38.018733 25177 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:38.018774 25177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:38.018867 25177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:38.019908 25177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "32a7072ca0a34b0ba017d9f146e6656f"
format_stamp: "Formatted at 2026-08-10 06:45:38 on dist-test-slave-vxj2"
server_key: "c7c5b8dae9fcf6841b2682a1f47d2ace"
server_key_iv: "3c1c3b153c13a85a66e9c0b2dc215f84"
server_key_version: "encryptionkey@0"
I20260810 06:45:38.020224 25177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "32a7072ca0a34b0ba017d9f146e6656f"
format_stamp: "Formatted at 2026-08-10 06:45:38 on dist-test-slave-vxj2"
server_key: "c7c5b8dae9fcf6841b2682a1f47d2ace"
server_key_iv: "3c1c3b153c13a85a66e9c0b2dc215f84"
server_key_version: "encryptionkey@0"
I20260810 06:45:38.021641 25177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:45:38.022378 25195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:38.022583 25177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:38.022672 25177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "32a7072ca0a34b0ba017d9f146e6656f"
format_stamp: "Formatted at 2026-08-10 06:45:38 on dist-test-slave-vxj2"
server_key: "c7c5b8dae9fcf6841b2682a1f47d2ace"
server_key_iv: "3c1c3b153c13a85a66e9c0b2dc215f84"
server_key_version: "encryptionkey@0"
I20260810 06:45:38.022728 25177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:38.037760 25177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:38.040179 25177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:38.040302 25177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:38.040802 25177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:38.040845 25177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:38.040869 25177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:38.040928 25177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[24737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344338, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
I20260810 06:45:38.048389 25177 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:38379
I20260810 06:45:38.048516 25328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:38379 every 8 connection(s)
I20260810 06:45:38.048839 25177 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:38.051673 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.041349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:36121 (local address 127.15.232.254:35699)
0810 06:45:38.041431 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.041434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.041658 (+ 224us) server_negotiation.cc:407] Connection header received
0810 06:45:38.042299 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.042300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.042315 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.042401 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:38.043301 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.044145 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.045305 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.045612 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.045694 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.048670 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:38.048672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:38.048672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:38.048681 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:38.050224 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:38.050647 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:38.050648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:38.050649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:38.050713 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:38.050986 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:38.050987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:38.050988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:38.051108 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:38.051184 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.051496 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.051583 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:45:38.051848 25204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.041329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35699 (local address 127.15.232.195:36121)
0810 06:45:38.041520 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.041560 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.042151 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.042496 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.042500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.042885 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:38.043186 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.043193 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.044266 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.044267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.045213 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.045217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.045745 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.045746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.045938 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.046538 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:38.046550 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:38.048550 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:38.050323 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:38.050329 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:38.050332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:38.050567 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:38.050785 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:38.050786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:38.050786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:38.050912 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:38.051207 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:38.051214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:38.051349 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.051458 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.051620 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260810 06:45:38.052363 25329 heartbeater.cc:347] Connected to a master server at 127.15.232.254:35699
I20260810 06:45:38.052452 25329 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:38.052640 25329 heartbeater.cc:511] Master 127.15.232.254:35699 requested a full tablet report, sending...
I20260810 06:45:38.053030 24786 ts_manager.cc:194] Registered new tserver with Master: 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195:38379)
I20260810 06:45:38.053540 24786 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:36121
I20260810 06:45:38.056156 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 25177
I20260810 06:45:38.056267 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:38.056483 16291 external_mini_cluster.cc:1428] Setting key edef92f0c3d6dcae310ca88bde5700e4
I20260810 06:45:38.057552 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:45:38.059532 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.053463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35699 (local address 127.15.232.195:55819)
0810 06:45:38.053614 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.053626 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.053686 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.053820 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.053821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.054000 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:38.054063 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.054065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.054707 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.054708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.055212 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.055213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.055603 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.055604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.055674 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.056080 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:38.056086 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:38.056499 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:38.058407 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:38.058408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:38.058409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:38.058603 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:38.058832 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:38.058833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:38.058833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:38.058897 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:38.059233 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:38.059234 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:38.059282 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.059359 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.059453 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260810 06:45:38.059613 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.053525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:55819 (local address 127.15.232.254:35699)
0810 06:45:38.053566 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.053568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.053674 (+ 106us) server_negotiation.cc:407] Connection header received
0810 06:45:38.053751 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.053752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.053762 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.053806 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:38.054141 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.054611 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.055310 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.055516 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.055600 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.056615 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:38.056617 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:38.056617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:38.056623 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:38.058301 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:38.058688 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:38.058688 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:38.058689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:38.058751 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:38.058964 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:38.058965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:38.058965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:38.059110 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:38.059179 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.059404 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.059488 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[24737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344338, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[24737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[24737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:45:38.082113 24843 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:38.076458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40600 (local address 127.15.232.254:35699)
0810 06:45:38.076551 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.076553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.076565 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:45:38.076607 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.076608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.076617 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.076678 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:38.077282 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.077819 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.078537 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.078713 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.078810 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.082005 (+ 3195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40600: BlockingRecv error: recv got EOF from 127.0.0.1:40600 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[24737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344338, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
I20260810 06:45:38.089656 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.082814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40614 (local address 127.15.232.254:35699)
0810 06:45:38.082871 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.082878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.082960 (+ 82us) server_negotiation.cc:407] Connection header received
0810 06:45:38.083001 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.083002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.083010 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.083063 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:38.083493 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.083961 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.084600 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.084772 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.084819 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.086931 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:38.086932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:38.086933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:38.086939 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:38.088607 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:38.088956 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:38.088957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:38.088957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:38.089044 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:38.089255 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:38.089256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:38.089256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:38.089357 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:38.089451 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.089535 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.089591 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:45:38.091287 24786 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:38.092821 24786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:38.099642 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.095283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:39153 (local address 127.0.0.1:40164)
0810 06:45:38.095480 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.095522 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.095590 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.095837 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.095840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.095843 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:38.095911 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.095913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.097216 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.097217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.098530 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.098533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.099268 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.099269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.099365 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.099379 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.099473 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.099529 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":68,"threads_started":1}
I20260810 06:45:38.099714 25353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.095284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:42115 (local address 127.0.0.1:46086)
0810 06:45:38.095591 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.095601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.095644 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.096549 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.096550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.096551 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:38.096580 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.096581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.097526 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.097527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.098479 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.098481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.099418 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.099419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.099492 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.099496 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.099528 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.099574 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":2,"thread_start_us":116,"threads_started":1}
I20260810 06:45:38.099743 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.095711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47228 (local address 127.15.232.195:38379)
0810 06:45:38.095921 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.095925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.096275 (+ 350us) server_negotiation.cc:407] Connection header received
0810 06:45:38.096319 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.096322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.096339 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.096403 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:38.096539 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.097501 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.098730 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.099391 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.099469 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.099528 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.099603 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.099646 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":106,"threads_started":1}
I20260810 06:45:38.100076 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.095501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38379 (local address 127.0.0.1:47228)
0810 06:45:38.096132 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.096141 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.096207 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.096413 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.096414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.096416 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:38.096459 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.096460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.097570 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.097571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.098653 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.098655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.099470 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.099472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.099526 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.099529 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.099590 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.099627 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":27,"thread_start_us":75,"threads_started":1}
I20260810 06:45:38.100111 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.095429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40164 (local address 127.15.232.194:39153)
0810 06:45:38.095642 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.095647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.095657 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:38.095699 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.095701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.095716 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.095810 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:38.096002 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.097102 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.098627 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.099174 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.099284 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.099346 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.099834 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.099894 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":93,"threads_started":1}
I20260810 06:45:38.100250 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.095416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46086 (local address 127.15.232.193:42115)
0810 06:45:38.095942 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.095945 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.095953 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:38.096006 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.096008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.096024 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.096559 (+ 535us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:38.096629 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.097435 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.098566 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.099298 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.099700 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.099768 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.099795 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.100129 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":92,"threads_started":1}
I20260810 06:45:38.100807 25087 tablet_service.cc:1467] Processing CreateTablet for tablet 6fc34ec86d2f496a92ba4abe1085c740 (DEFAULT_TABLE table=test-table [id=cc90930cfdfa4266b5f4533ff98dccb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:38.100800 25253 tablet_service.cc:1467] Processing CreateTablet for tablet 6fc34ec86d2f496a92ba4abe1085c740 (DEFAULT_TABLE table=test-table [id=cc90930cfdfa4266b5f4533ff98dccb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:38.101087 25253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fc34ec86d2f496a92ba4abe1085c740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.101089 25087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fc34ec86d2f496a92ba4abe1085c740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.101799 24936 tablet_service.cc:1467] Processing CreateTablet for tablet 6fc34ec86d2f496a92ba4abe1085c740 (DEFAULT_TABLE table=test-table [id=cc90930cfdfa4266b5f4533ff98dccb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:38.102135 24936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fc34ec86d2f496a92ba4abe1085c740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.104621 25359 tablet_bootstrap.cc:492] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe: Bootstrap starting.
I20260810 06:45:38.104854 25360 tablet_bootstrap.cc:492] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f: Bootstrap starting.
I20260810 06:45:38.105459 25359 tablet_bootstrap.cc:654] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.105468 25360 tablet_bootstrap.cc:654] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.105822 25359 log.cc:824] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:38.105855 25360 log.cc:824] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:38.105993 25361 tablet_bootstrap.cc:492] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5: Bootstrap starting.
I20260810 06:45:38.106647 25359 tablet_bootstrap.cc:492] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe: No bootstrap required, opened a new log
I20260810 06:45:38.106719 25361 tablet_bootstrap.cc:654] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.106926 25359 ts_tablet_manager.cc:1397] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:45:38.106968 25360 tablet_bootstrap.cc:492] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f: No bootstrap required, opened a new log
I20260810 06:45:38.107024 25360 ts_tablet_manager.cc:1397] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:45:38.107213 25361 log.cc:824] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:38.107879 25361 tablet_bootstrap.cc:492] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5: No bootstrap required, opened a new log
I20260810 06:45:38.107959 25361 ts_tablet_manager.cc:1397] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:45:38.109061 25360 raft_consensus.cc:348] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.109061 25359 raft_consensus.cc:348] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.109272 25359 raft_consensus.cc:374] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.109272 25360 raft_consensus.cc:374] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.109313 25359 raft_consensus.cc:729] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e68488536e348e88ee3491a468288fe, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.109313 25360 raft_consensus.cc:729] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a7072ca0a34b0ba017d9f146e6656f, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.109452 25359 consensus_queue.cc:260] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.109441 25360 consensus_queue.cc:260] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.109483 25361 raft_consensus.cc:348] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.109656 25361 raft_consensus.cc:374] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.109709 25361 raft_consensus.cc:729] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4db1bb333a7141ee9e342a77941df9f5, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.109719 25359 ts_tablet_manager.cc:1428] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:38.109719 25360 ts_tablet_manager.cc:1428] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:38.109846 25329 heartbeater.cc:503] Master 127.15.232.254:35699 was elected leader, sending a full tablet report...
I20260810 06:45:38.110163 25166 heartbeater.cc:503] Master 127.15.232.254:35699 was elected leader, sending a full tablet report...
I20260810 06:45:38.110474 25361 consensus_queue.cc:260] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.110780 25009 heartbeater.cc:503] Master 127.15.232.254:35699 was elected leader, sending a full tablet report...
I20260810 06:45:38.110775 25361 ts_tablet_manager.cc:1428] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:45:38.135105 25366 raft_consensus.cc:482] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:38.135232 25366 raft_consensus.cc:504] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.135675 25366 leader_election.cc:288] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153), 4db1bb333a7141ee9e342a77941df9f5 (127.15.232.193:42115)
W20260810 06:45:38.135974 25167 tablet.cc:2369] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:45:38.138829 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.135971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:45623 (local address 127.15.232.194:39153)
0810 06:45:38.136017 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.136019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.136245 (+ 226us) server_negotiation.cc:407] Connection header received
0810 06:45:38.136317 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.136318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.136327 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.136427 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:38.136630 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.137093 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.138097 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.138465 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.138550 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.138559 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.138724 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.138764 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:45:38.138842 25369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.135913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:39153 (local address 127.15.232.195:45623)
0810 06:45:38.136175 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.136187 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.136249 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.136443 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.136444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.136446 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:38.136530 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.136532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.137194 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.137195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.138000 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.138002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.138563 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.138564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.138647 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.138650 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.138718 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.138757 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":86,"thread_start_us":121,"threads_started":1}
I20260810 06:45:38.138926 25204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.135912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:42115 (local address 127.15.232.195:37617)
0810 06:45:38.135970 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.135984 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.136045 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.136366 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.136368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.136370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:38.136452 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.136454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.137275 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.137276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.138011 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.138013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.138600 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.138601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.138694 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.138697 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.138778 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.138824 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:45:38.138906 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.136167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:37617 (local address 127.15.232.193:42115)
0810 06:45:38.136214 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.136216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.136225 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:38.136260 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.136266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.136276 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.136357 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:38.136539 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.137108 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.138115 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.138495 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.138588 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.138596 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.138785 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.138825 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:45:38.139310 24959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fc34ec86d2f496a92ba4abe1085c740" candidate_uuid: "32a7072ca0a34b0ba017d9f146e6656f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4db1bb333a7141ee9e342a77941df9f5" is_pre_election: true
I20260810 06:45:38.139366 25109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fc34ec86d2f496a92ba4abe1085c740" candidate_uuid: "32a7072ca0a34b0ba017d9f146e6656f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68488536e348e88ee3491a468288fe" is_pre_election: true
I20260810 06:45:38.139498 24959 raft_consensus.cc:2454] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a7072ca0a34b0ba017d9f146e6656f in term 0.
I20260810 06:45:38.139497 25109 raft_consensus.cc:2454] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a7072ca0a34b0ba017d9f146e6656f in term 0.
I20260810 06:45:38.139770 25198 leader_election.cc:302] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a7072ca0a34b0ba017d9f146e6656f, 6e68488536e348e88ee3491a468288fe; no voters:
I20260810 06:45:38.139916 25366 raft_consensus.cc:2781] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:38.140018 25366 raft_consensus.cc:482] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:38.140054 25366 raft_consensus.cc:3037] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.140980 25366 raft_consensus.cc:504] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.141131 25366 leader_election.cc:288] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [CANDIDATE]: Term 1 election: Requested vote from peers 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153), 4db1bb333a7141ee9e342a77941df9f5 (127.15.232.193:42115)
I20260810 06:45:38.141345 25109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fc34ec86d2f496a92ba4abe1085c740" candidate_uuid: "32a7072ca0a34b0ba017d9f146e6656f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68488536e348e88ee3491a468288fe"
I20260810 06:45:38.141453 25109 raft_consensus.cc:3037] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.142267 25109 raft_consensus.cc:2454] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32a7072ca0a34b0ba017d9f146e6656f in term 1.
I20260810 06:45:38.142416 24959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fc34ec86d2f496a92ba4abe1085c740" candidate_uuid: "32a7072ca0a34b0ba017d9f146e6656f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4db1bb333a7141ee9e342a77941df9f5"
I20260810 06:45:38.142467 25198 leader_election.cc:302] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a7072ca0a34b0ba017d9f146e6656f, 6e68488536e348e88ee3491a468288fe; no voters:
I20260810 06:45:38.142514 24959 raft_consensus.cc:3037] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.142554 25366 raft_consensus.cc:2781] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:38.142704 25366 raft_consensus.cc:686] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 LEADER]: Becoming Leader. State: Replica: 32a7072ca0a34b0ba017d9f146e6656f, State: Running, Role: LEADER
I20260810 06:45:38.142817 25366 consensus_queue.cc:237] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.143594 24959 raft_consensus.cc:2454] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32a7072ca0a34b0ba017d9f146e6656f in term 1.
I20260810 06:45:38.144255 24786 catalog_manager.cc:5495] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f reported cstate change: term changed from 0 to 1, leader changed from <none> to 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195). New cstate: current_term: 1 leader_uuid: "32a7072ca0a34b0ba017d9f146e6656f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } health_report { overall_health: HEALTHY } } }
I20260810 06:45:38.158103 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.155558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47242 (local address 127.15.232.195:38379)
0810 06:45:38.155629 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.155632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.155641 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:38.155708 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.155709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.155725 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.155805 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:38.155947 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.156419 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.157312 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.157498 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.157573 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.157741 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:38.157841 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:38.157972 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.157991 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.158013 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:45:38.159960 25109 raft_consensus.cc:1261] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Refusing update from remote peer 32a7072ca0a34b0ba017d9f146e6656f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:38.159993 24959 raft_consensus.cc:1261] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Refusing update from remote peer 32a7072ca0a34b0ba017d9f146e6656f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:38.160310 25366 consensus_queue.cc:1035] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:38.160473 25370 consensus_queue.cc:1035] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:38.165395 24786 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:38.165498 24786 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:38.166211 24786 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9: Sending DeleteTablet for 3 replicas of tablet 6fc34ec86d2f496a92ba4abe1085c740
I20260810 06:45:38.166543 25087 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fc34ec86d2f496a92ba4abe1085c740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:38 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:40164
I20260810 06:45:38.166591 25253 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fc34ec86d2f496a92ba4abe1085c740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:38 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:47228
I20260810 06:45:38.166585 24936 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fc34ec86d2f496a92ba4abe1085c740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:38 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:46086
I20260810 06:45:38.166815 25387 tablet_replica.cc:330] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5: stopping tablet replica
I20260810 06:45:38.166923 25387 raft_consensus.cc:2229] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:38.167032 25387 raft_consensus.cc:2258] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:38.167112 25385 tablet_replica.cc:330] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f: stopping tablet replica
I20260810 06:45:38.167057 24786 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:38.167196 24786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:38.167219 25385 raft_consensus.cc:2229] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:38.167333 25385 raft_consensus.cc:2258] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:38.167704 25384 tablet_replica.cc:330] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe: stopping tablet replica
I20260810 06:45:38.168200 25384 raft_consensus.cc:2229] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:38.168327 25384 raft_consensus.cc:2258] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:38.168613 24936 tablet_service.cc:1467] Processing CreateTablet for tablet 741cad995d214931b7976a7630ebd503 (DEFAULT_TABLE table=test-table [id=2abd462f23c546ae99415ad677af94b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:38.168668 25087 tablet_service.cc:1467] Processing CreateTablet for tablet 741cad995d214931b7976a7630ebd503 (DEFAULT_TABLE table=test-table [id=2abd462f23c546ae99415ad677af94b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:38.168764 25087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 741cad995d214931b7976a7630ebd503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.168766 24936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 741cad995d214931b7976a7630ebd503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.168730 25253 tablet_service.cc:1467] Processing CreateTablet for tablet 741cad995d214931b7976a7630ebd503 (DEFAULT_TABLE table=test-table [id=2abd462f23c546ae99415ad677af94b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:38.168964 25253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 741cad995d214931b7976a7630ebd503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.170208 25359 tablet_bootstrap.cc:492] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe: Bootstrap starting.
I20260810 06:45:38.170326 25360 tablet_bootstrap.cc:492] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f: Bootstrap starting.
I20260810 06:45:38.170630 25359 tablet_bootstrap.cc:654] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.170768 25360 tablet_bootstrap.cc:654] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.171023 25361 tablet_bootstrap.cc:492] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5: Bootstrap starting.
I20260810 06:45:38.168051 25385 ts_tablet_manager.cc:1905] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:38.171397 25359 tablet_bootstrap.cc:492] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe: No bootstrap required, opened a new log
I20260810 06:45:38.171443 25359 ts_tablet_manager.cc:1397] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:38.171612 25359 raft_consensus.cc:348] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.171687 25359 raft_consensus.cc:374] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.171710 25359 raft_consensus.cc:729] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e68488536e348e88ee3491a468288fe, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.171712 25361 tablet_bootstrap.cc:654] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.171759 25359 consensus_queue.cc:260] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.171885 25359 ts_tablet_manager.cc:1428] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:38.172551 25361 tablet_bootstrap.cc:492] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5: No bootstrap required, opened a new log
I20260810 06:45:38.172598 25361 ts_tablet_manager.cc:1397] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:45:38.172748 25361 raft_consensus.cc:348] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.172817 25361 raft_consensus.cc:374] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.172835 25361 raft_consensus.cc:729] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4db1bb333a7141ee9e342a77941df9f5, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.172904 25361 consensus_queue.cc:260] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.172993 25361 ts_tablet_manager.cc:1428] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:38.173779 25387 ts_tablet_manager.cc:1905] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:38.173815 25385 ts_tablet_manager.cc:1918] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:38.173986 25385 log.cc:1196] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/6fc34ec86d2f496a92ba4abe1085c740
I20260810 06:45:38.174031 25384 ts_tablet_manager.cc:1905] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:38.174404 25385 ts_tablet_manager.cc:1939] T 6fc34ec86d2f496a92ba4abe1085c740 P 32a7072ca0a34b0ba017d9f146e6656f: Deleting consensus metadata
I20260810 06:45:38.174872 24773 catalog_manager.cc:4841] TS 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195:38379): tablet 6fc34ec86d2f496a92ba4abe1085c740 (table test-table [id=cc90930cfdfa4266b5f4533ff98dccb6]) successfully deleted
I20260810 06:45:38.175268 25387 ts_tablet_manager.cc:1918] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:38.175346 25387 log.cc:1196] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/wal/wals/6fc34ec86d2f496a92ba4abe1085c740
I20260810 06:45:38.175596 25387 ts_tablet_manager.cc:1939] T 6fc34ec86d2f496a92ba4abe1085c740 P 4db1bb333a7141ee9e342a77941df9f5: Deleting consensus metadata
I20260810 06:45:38.175818 25384 ts_tablet_manager.cc:1918] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:38.175895 25384 log.cc:1196] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/6fc34ec86d2f496a92ba4abe1085c740
I20260810 06:45:38.176064 25360 tablet_bootstrap.cc:492] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f: No bootstrap required, opened a new log
I20260810 06:45:38.176113 25360 ts_tablet_manager.cc:1397] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260810 06:45:38.176173 25384 ts_tablet_manager.cc:1939] T 6fc34ec86d2f496a92ba4abe1085c740 P 6e68488536e348e88ee3491a468288fe: Deleting consensus metadata
I20260810 06:45:38.176179 24774 catalog_manager.cc:4841] TS 4db1bb333a7141ee9e342a77941df9f5 (127.15.232.193:42115): tablet 6fc34ec86d2f496a92ba4abe1085c740 (table test-table [id=cc90930cfdfa4266b5f4533ff98dccb6]) successfully deleted
I20260810 06:45:38.176245 25360 raft_consensus.cc:348] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.176306 25360 raft_consensus.cc:374] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.176333 25360 raft_consensus.cc:729] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a7072ca0a34b0ba017d9f146e6656f, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.176378 25360 consensus_queue.cc:260] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.176465 25360 ts_tablet_manager.cc:1428] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:38.176667 24773 catalog_manager.cc:4841] TS 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153): tablet 6fc34ec86d2f496a92ba4abe1085c740 (table test-table [id=cc90930cfdfa4266b5f4533ff98dccb6]) successfully deleted
I20260810 06:45:38.196249 25368 raft_consensus.cc:482] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:38.196336 25368 raft_consensus.cc:504] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.196679 25368 leader_election.cc:288] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153), 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195:38379)
I20260810 06:45:38.200023 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.197071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:39409 (local address 127.15.232.195:38379)
0810 06:45:38.197112 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.197114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.197122 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:38.197194 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.197195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.197205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.197257 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:38.197425 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.197911 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.199245 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.199692 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.199729 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.199738 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.199913 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.199948 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:45:38.200043 25391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.196923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38379 (local address 127.15.232.193:39409)
0810 06:45:38.197044 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.197055 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.197106 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.197266 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.197267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.197268 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:38.197348 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.197350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.198001 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.198002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.199130 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.199133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.199784 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.199785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.199838 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.199840 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.199908 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.199948 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260810 06:45:38.200409 25276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "741cad995d214931b7976a7630ebd503" candidate_uuid: "4db1bb333a7141ee9e342a77941df9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a7072ca0a34b0ba017d9f146e6656f" is_pre_election: true
I20260810 06:45:38.200563 25276 raft_consensus.cc:2454] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4db1bb333a7141ee9e342a77941df9f5 in term 0.
I20260810 06:45:38.200580 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.196827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:39153 (local address 127.15.232.193:39877)
0810 06:45:38.196892 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.196908 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.196972 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.197199 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.197200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.197202 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:38.197331 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.197334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.198368 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.198369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.199387 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.199389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.200300 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.200301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.200396 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.200401 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.200460 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.200503 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 06:45:38.200603 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.196913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:39877 (local address 127.15.232.194:39153)
0810 06:45:38.196964 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.196966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.196976 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:38.197078 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.197079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.197090 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.197143 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:38.197439 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.198271 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.199483 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.200120 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.200200 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.200209 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.200485 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.200537 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:45:38.200773 24885 leader_election.cc:302] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a7072ca0a34b0ba017d9f146e6656f, 4db1bb333a7141ee9e342a77941df9f5; no voters:
I20260810 06:45:38.200909 25368 raft_consensus.cc:2781] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:38.200871 25109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "741cad995d214931b7976a7630ebd503" candidate_uuid: "4db1bb333a7141ee9e342a77941df9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68488536e348e88ee3491a468288fe" is_pre_election: true
I20260810 06:45:38.200941 25368 raft_consensus.cc:482] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:38.200966 25368 raft_consensus.cc:3037] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.200969 25109 raft_consensus.cc:2454] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4db1bb333a7141ee9e342a77941df9f5 in term 0.
I20260810 06:45:38.201984 25368 raft_consensus.cc:504] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.202147 25368 leader_election.cc:288] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [CANDIDATE]: Term 1 election: Requested vote from peers 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153), 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195:38379)
I20260810 06:45:38.202318 25109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "741cad995d214931b7976a7630ebd503" candidate_uuid: "4db1bb333a7141ee9e342a77941df9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68488536e348e88ee3491a468288fe"
I20260810 06:45:38.202430 25109 raft_consensus.cc:3037] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.202428 25276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "741cad995d214931b7976a7630ebd503" candidate_uuid: "4db1bb333a7141ee9e342a77941df9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a7072ca0a34b0ba017d9f146e6656f"
I20260810 06:45:38.202517 25276 raft_consensus.cc:3037] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.203142 25109 raft_consensus.cc:2454] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4db1bb333a7141ee9e342a77941df9f5 in term 1.
I20260810 06:45:38.203316 24885 leader_election.cc:302] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4db1bb333a7141ee9e342a77941df9f5, 6e68488536e348e88ee3491a468288fe; no voters:
I20260810 06:45:38.203405 25368 raft_consensus.cc:2781] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:38.203433 25276 raft_consensus.cc:2454] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4db1bb333a7141ee9e342a77941df9f5 in term 1.
I20260810 06:45:38.203548 25368 raft_consensus.cc:686] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4db1bb333a7141ee9e342a77941df9f5, State: Running, Role: LEADER
I20260810 06:45:38.203624 25368 consensus_queue.cc:237] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.204419 24785 catalog_manager.cc:5495] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4db1bb333a7141ee9e342a77941df9f5 (127.15.232.193). New cstate: current_term: 1 leader_uuid: "4db1bb333a7141ee9e342a77941df9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } health_report { overall_health: UNKNOWN } } }
W20260810 06:45:38.210172 25010 tablet.cc:2369] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[24737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344337, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.254@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
I20260810 06:45:38.234697 25399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.225952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35699 (local address 127.0.0.1:40622)
0810 06:45:38.226157 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.226166 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.226227 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.226387 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.226388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.226703 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:38.226793 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.226795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.227868 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.227869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.228414 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.228415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.228945 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.228957 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.229011 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.229693 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:38.229698 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:38.231472 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:38.233425 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:38.233435 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:38.233437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:38.233655 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:38.233959 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:38.233960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:38.233961 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:38.234047 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:38.234401 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:38.234409 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:38.234455 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.234516 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.234620 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":103,"threads_started":1}
I20260810 06:45:38.234824 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.226016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40622 (local address 127.15.232.254:35699)
0810 06:45:38.226092 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.226095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.226237 (+ 142us) server_negotiation.cc:407] Connection header received
0810 06:45:38.226274 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.226275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.226289 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.226374 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:38.226888 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.227772 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.228499 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.228843 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.228922 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.231573 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:38.231575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:38.231575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:38.231583 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:38.233327 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:38.233741 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:38.233742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:38.233743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:38.233824 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:38.234156 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:38.234157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:38.234157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:38.234313 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:38.234397 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.234570 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.234694 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":47}
I20260810 06:45:38.237072 24785 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:40622:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:45:38.237383 24785 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:38.239035 24936 tablet_service.cc:1467] Processing CreateTablet for tablet b3faf1e7f7a8413694d1d7ab46a58f59 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d54b950b2907497fafd367f112816a39]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:45:38.239042 25087 tablet_service.cc:1467] Processing CreateTablet for tablet b3faf1e7f7a8413694d1d7ab46a58f59 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d54b950b2907497fafd367f112816a39]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:45:38.239068 25253 tablet_service.cc:1467] Processing CreateTablet for tablet b3faf1e7f7a8413694d1d7ab46a58f59 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d54b950b2907497fafd367f112816a39]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:45:38.239203 24936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3faf1e7f7a8413694d1d7ab46a58f59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.239203 25253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3faf1e7f7a8413694d1d7ab46a58f59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.239208 25087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3faf1e7f7a8413694d1d7ab46a58f59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.240736 25360 tablet_bootstrap.cc:492] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f: Bootstrap starting.
I20260810 06:45:38.240734 25361 tablet_bootstrap.cc:492] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: Bootstrap starting.
I20260810 06:45:38.240939 25359 tablet_bootstrap.cc:492] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe: Bootstrap starting.
I20260810 06:45:38.241066 25360 tablet_bootstrap.cc:654] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.241084 25361 tablet_bootstrap.cc:654] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.241429 25359 tablet_bootstrap.cc:654] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.241854 25361 tablet_bootstrap.cc:492] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: No bootstrap required, opened a new log
I20260810 06:45:38.241917 25361 ts_tablet_manager.cc:1397] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:38.242081 25360 tablet_bootstrap.cc:492] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f: No bootstrap required, opened a new log
I20260810 06:45:38.242118 25360 ts_tablet_manager.cc:1397] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:38.242072 25361 raft_consensus.cc:348] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.242148 25361 raft_consensus.cc:374] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.242173 25361 raft_consensus.cc:729] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4db1bb333a7141ee9e342a77941df9f5, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.242215 25361 consensus_queue.cc:260] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.242221 25360 raft_consensus.cc:348] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.242281 25360 raft_consensus.cc:374] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.242317 25360 raft_consensus.cc:729] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a7072ca0a34b0ba017d9f146e6656f, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.242311 25361 ts_tablet_manager.cc:1428] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 06:45:38.242360 25360 consensus_queue.cc:260] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.242453 25360 ts_tablet_manager.cc:1428] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:38.242472 25368 tablet_replica.cc:439] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } } }
I20260810 06:45:38.242527 25359 tablet_bootstrap.cc:492] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe: No bootstrap required, opened a new log
I20260810 06:45:38.242547 25368 tablet_replica.cc:442] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:38.242578 25359 ts_tablet_manager.cc:1397] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 06:45:38.242547 25370 tablet_replica.cc:439] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } } }
I20260810 06:45:38.242616 25370 tablet_replica.cc:442] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:38.242748 25359 raft_consensus.cc:348] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.242851 25359 raft_consensus.cc:374] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.242877 25359 raft_consensus.cc:729] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e68488536e348e88ee3491a468288fe, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.242935 25359 consensus_queue.cc:260] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.243033 25359 ts_tablet_manager.cc:1428] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 06:45:38.243559 25367 tablet_replica.cc:439] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } } }
I20260810 06:45:38.243683 25367 tablet_replica.cc:442] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:38.256696 25368 raft_consensus.cc:482] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:38.256808 25368 raft_consensus.cc:504] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.256975 25368 leader_election.cc:288] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153), 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195:38379)
I20260810 06:45:38.257267 25109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3faf1e7f7a8413694d1d7ab46a58f59" candidate_uuid: "4db1bb333a7141ee9e342a77941df9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68488536e348e88ee3491a468288fe" is_pre_election: true
I20260810 06:45:38.257267 25276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3faf1e7f7a8413694d1d7ab46a58f59" candidate_uuid: "4db1bb333a7141ee9e342a77941df9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a7072ca0a34b0ba017d9f146e6656f" is_pre_election: true
I20260810 06:45:38.257376 25109 raft_consensus.cc:2454] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4db1bb333a7141ee9e342a77941df9f5 in term 0.
I20260810 06:45:38.257376 25276 raft_consensus.cc:2454] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4db1bb333a7141ee9e342a77941df9f5 in term 0.
I20260810 06:45:38.257551 24885 leader_election.cc:302] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4db1bb333a7141ee9e342a77941df9f5, 6e68488536e348e88ee3491a468288fe; no voters:
I20260810 06:45:38.257686 25368 raft_consensus.cc:2781] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:38.257731 25368 raft_consensus.cc:482] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:38.257769 25368 raft_consensus.cc:3037] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.258507 25368 raft_consensus.cc:504] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.258651 25368 leader_election.cc:288] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [CANDIDATE]: Term 1 election: Requested vote from peers 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153), 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195:38379)
I20260810 06:45:38.258818 25109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3faf1e7f7a8413694d1d7ab46a58f59" candidate_uuid: "4db1bb333a7141ee9e342a77941df9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68488536e348e88ee3491a468288fe"
I20260810 06:45:38.258917 25109 raft_consensus.cc:3037] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.258908 25276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3faf1e7f7a8413694d1d7ab46a58f59" candidate_uuid: "4db1bb333a7141ee9e342a77941df9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a7072ca0a34b0ba017d9f146e6656f"
I20260810 06:45:38.258982 25276 raft_consensus.cc:3037] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.259793 25109 raft_consensus.cc:2454] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4db1bb333a7141ee9e342a77941df9f5 in term 1.
I20260810 06:45:38.260175 25276 raft_consensus.cc:2454] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4db1bb333a7141ee9e342a77941df9f5 in term 1.
I20260810 06:45:38.260408 24885 leader_election.cc:302] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4db1bb333a7141ee9e342a77941df9f5, 6e68488536e348e88ee3491a468288fe; no voters:
I20260810 06:45:38.260561 25368 raft_consensus.cc:2781] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:38.260677 25368 raft_consensus.cc:686] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4db1bb333a7141ee9e342a77941df9f5, State: Running, Role: LEADER
I20260810 06:45:38.260732 25368 consensus_queue.cc:237] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.261018 25392 tablet_replica.cc:439] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: TxnStatusTablet state changed. Reason: New leader 4db1bb333a7141ee9e342a77941df9f5. Latest consensus state: current_term: 1 leader_uuid: "4db1bb333a7141ee9e342a77941df9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } } }
I20260810 06:45:38.261125 25392 tablet_replica.cc:442] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:45:38.261400 25405 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:45:38.261546 25405 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:45:38.261680 24786 catalog_manager.cc:5495] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4db1bb333a7141ee9e342a77941df9f5 (127.15.232.193). New cstate: current_term: 1 leader_uuid: "4db1bb333a7141ee9e342a77941df9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } health_report { overall_health: UNKNOWN } } }
I20260810 06:45:38.268451 25399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.264892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:42115 (local address 127.0.0.1:46092)
0810 06:45:38.264955 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.264969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.265024 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.265277 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.265278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.265279 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:38.265332 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.265334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.266322 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.266324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.267268 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.267272 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.267815 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.267816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.267893 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.267911 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:38.268251 (+ 340us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:38.268262 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.268313 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.268365 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260810 06:45:38.268451 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.265034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46092 (local address 127.15.232.193:42115)
0810 06:45:38.265079 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.265081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.265090 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:38.265127 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.265128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.265172 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.265271 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:38.265417 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.266218 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.267375 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.267707 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.267807 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.268026 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:38.268158 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:38.268250 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.268330 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.268353 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:45:38.269471 25109 raft_consensus.cc:1261] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Refusing update from remote peer 4db1bb333a7141ee9e342a77941df9f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:38.269526 25276 raft_consensus.cc:1261] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Refusing update from remote peer 4db1bb333a7141ee9e342a77941df9f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:38.269788 25392 consensus_queue.cc:1035] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:38.269995 25368 consensus_queue.cc:1035] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:38.270483 25367 tablet_replica.cc:439] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe: TxnStatusTablet state changed. Reason: New leader 4db1bb333a7141ee9e342a77941df9f5. Latest consensus state: current_term: 1 leader_uuid: "4db1bb333a7141ee9e342a77941df9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } } }
I20260810 06:45:38.270573 25367 tablet_replica.cc:442] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:38.270632 25370 tablet_replica.cc:439] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f: TxnStatusTablet state changed. Reason: New leader 4db1bb333a7141ee9e342a77941df9f5. Latest consensus state: current_term: 1 leader_uuid: "4db1bb333a7141ee9e342a77941df9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } } }
I20260810 06:45:38.270714 25370 tablet_replica.cc:442] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:38.271494 25368 tablet_replica.cc:439] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4db1bb333a7141ee9e342a77941df9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } } }
I20260810 06:45:38.271581 25368 tablet_replica.cc:442] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:45:38.271654 25374 mvcc.cc:205] Tried to move back new op lower bound from 7316866409549803520 to 7316866409516400640. Current Snapshot: MvccSnapshot[applied={T|T < 7316866409549803520 or (T in {7316866409549803520})}]
I20260810 06:45:38.272094 25370 tablet_replica.cc:439] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4db1bb333a7141ee9e342a77941df9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } } }
I20260810 06:45:38.272176 25370 tablet_replica.cc:442] T b3faf1e7f7a8413694d1d7ab46a58f59 P 32a7072ca0a34b0ba017d9f146e6656f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:38.273896 25392 tablet_replica.cc:439] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4db1bb333a7141ee9e342a77941df9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } } }
I20260810 06:45:38.274000 25392 tablet_replica.cc:442] T b3faf1e7f7a8413694d1d7ab46a58f59 P 4db1bb333a7141ee9e342a77941df9f5: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:45:38.274319 25375 mvcc.cc:205] Tried to move back new op lower bound from 7316866409549803520 to 7316866409516400640. Current Snapshot: MvccSnapshot[applied={T|T < 7316866409549803520}]
I20260810 06:45:38.274425 25367 tablet_replica.cc:439] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4db1bb333a7141ee9e342a77941df9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } } }
I20260810 06:45:38.274490 25367 tablet_replica.cc:442] T b3faf1e7f7a8413694d1d7ab46a58f59 P 6e68488536e348e88ee3491a468288fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:45:38.277211 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.273768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46098 (local address 127.15.232.193:42115)
0810 06:45:38.273819 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.273821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.273831 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:38.273854 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.273855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.273866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.273926 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:38.274181 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.274983 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.276234 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.276532 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.276607 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.276819 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:38.276855 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:38.276906 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.277092 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.277113 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:45:38.283526 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.280017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:40249 (local address 127.15.232.193:42115)
0810 06:45:38.280130 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.280132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.280175 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:45:38.280255 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.280256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.280267 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.280340 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:38.280469 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.281288 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.282584 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.282927 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.283055 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.283229 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:38.283263 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:38.283327 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.283392 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.283425 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":92}
I20260810 06:45:38.283550 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.279968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:42115 (local address 127.15.232.193:40249)
0810 06:45:38.280116 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.280127 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.280187 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.280340 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.280341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.280342 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:38.280398 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.280400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.281392 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.281393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.282467 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.282470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.283053 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.283054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.283123 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.283138 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:38.283335 (+ 197us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:38.283337 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.283387 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.283431 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":90,"threads_started":1}
I20260810 06:45:38.286233 25276 raft_consensus.cc:1261] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Refusing update from remote peer 4db1bb333a7141ee9e342a77941df9f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:38.286381 25108 raft_consensus.cc:1261] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Refusing update from remote peer 4db1bb333a7141ee9e342a77941df9f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:38.286641 25392 consensus_queue.cc:1035] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:38.286818 25392 consensus_queue.cc:1035] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:38.303082 24785 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:38.303202 24785 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:38.303807 24785 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9: Sending DeleteTablet for 3 replicas of tablet 741cad995d214931b7976a7630ebd503
I20260810 06:45:38.304057 25087 tablet_service.cc:1514] Processing DeleteTablet for tablet 741cad995d214931b7976a7630ebd503 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:38 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:40164
I20260810 06:45:38.304088 25253 tablet_service.cc:1514] Processing DeleteTablet for tablet 741cad995d214931b7976a7630ebd503 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:38 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:47228
I20260810 06:45:38.304174 25385 tablet_replica.cc:330] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f: stopping tablet replica
I20260810 06:45:38.304177 25384 tablet_replica.cc:330] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe: stopping tablet replica
I20260810 06:45:38.304240 25385 raft_consensus.cc:2229] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:38.304244 25384 raft_consensus.cc:2229] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:38.304278 25384 raft_consensus.cc:2258] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:38.304278 25385 raft_consensus.cc:2258] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:38.304661 24936 tablet_service.cc:1514] Processing DeleteTablet for tablet 741cad995d214931b7976a7630ebd503 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:38 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:46086
I20260810 06:45:38.304741 25387 tablet_replica.cc:330] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5: stopping tablet replica
I20260810 06:45:38.304782 25387 raft_consensus.cc:2229] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:38.304857 25387 raft_consensus.cc:2258] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:38.305219 25387 ts_tablet_manager.cc:1905] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:38.307107 25387 ts_tablet_manager.cc:1918] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:38.307163 25387 log.cc:1196] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/wal/wals/741cad995d214931b7976a7630ebd503
I20260810 06:45:38.307395 25387 ts_tablet_manager.cc:1939] T 741cad995d214931b7976a7630ebd503 P 4db1bb333a7141ee9e342a77941df9f5: Deleting consensus metadata
I20260810 06:45:38.308629 24774 catalog_manager.cc:4841] TS 4db1bb333a7141ee9e342a77941df9f5 (127.15.232.193:42115): tablet 741cad995d214931b7976a7630ebd503 (table test-table [id=2abd462f23c546ae99415ad677af94b5]) successfully deleted
I20260810 06:45:38.308744 25385 ts_tablet_manager.cc:1905] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:38.310341 25385 ts_tablet_manager.cc:1918] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:38.310410 25385 log.cc:1196] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/741cad995d214931b7976a7630ebd503
I20260810 06:45:38.310623 25385 ts_tablet_manager.cc:1939] T 741cad995d214931b7976a7630ebd503 P 32a7072ca0a34b0ba017d9f146e6656f: Deleting consensus metadata
I20260810 06:45:38.310992 25384 ts_tablet_manager.cc:1905] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:38.311046 24773 catalog_manager.cc:4841] TS 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195:38379): tablet 741cad995d214931b7976a7630ebd503 (table test-table [id=2abd462f23c546ae99415ad677af94b5]) successfully deleted
I20260810 06:45:38.312587 25384 ts_tablet_manager.cc:1918] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:38.312654 25384 log.cc:1196] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/741cad995d214931b7976a7630ebd503
I20260810 06:45:38.312861 25384 ts_tablet_manager.cc:1939] T 741cad995d214931b7976a7630ebd503 P 6e68488536e348e88ee3491a468288fe: Deleting consensus metadata
I20260810 06:45:38.313223 24773 catalog_manager.cc:4841] TS 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153): tablet 741cad995d214931b7976a7630ebd503 (table test-table [id=2abd462f23c546ae99415ad677af94b5]) successfully deleted
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[24737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[24737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[24737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:45:38.338527 24843 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:38.332074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40624 (local address 127.15.232.254:35699)
0810 06:45:38.332145 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.332148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.332278 (+ 130us) server_negotiation.cc:407] Connection header received
0810 06:45:38.332339 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.332340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.332351 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.332426 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:38.333117 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.333928 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.334743 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.335060 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.335136 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.338424 (+ 3288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40624: BlockingRecv error: recv got EOF from 127.0.0.1:40624 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[24737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM
I20260810 06:45:38.347294 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.339368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40632 (local address 127.15.232.254:35699)
0810 06:45:38.339414 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.339415 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.339541 (+ 126us) server_negotiation.cc:407] Connection header received
0810 06:45:38.339632 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.339633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.339645 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.339709 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:38.340213 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.340738 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.341646 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.341870 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.341964 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.344350 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:38.344351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:38.344352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:38.344361 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:38.345993 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:38.346392 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:38.346394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:38.346394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:38.346473 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:38.346704 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:38.346705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:38.346705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:38.346832 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:38.346949 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.347097 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.347182 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:45:38.348953 24785 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:38.349205 24785 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:38.350780 25087 tablet_service.cc:1467] Processing CreateTablet for tablet e5e6e48e2a134371b0056814dbd02d41 (DEFAULT_TABLE table=test-table [id=8b469ddd8fa04626b1dda51026c888f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:38.350831 24936 tablet_service.cc:1467] Processing CreateTablet for tablet e5e6e48e2a134371b0056814dbd02d41 (DEFAULT_TABLE table=test-table [id=8b469ddd8fa04626b1dda51026c888f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:38.350858 25253 tablet_service.cc:1467] Processing CreateTablet for tablet e5e6e48e2a134371b0056814dbd02d41 (DEFAULT_TABLE table=test-table [id=8b469ddd8fa04626b1dda51026c888f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:38.350947 25087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e6e48e2a134371b0056814dbd02d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.350979 25253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e6e48e2a134371b0056814dbd02d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.350984 24936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e6e48e2a134371b0056814dbd02d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.352532 25360 tablet_bootstrap.cc:492] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f: Bootstrap starting.
I20260810 06:45:38.352626 25361 tablet_bootstrap.cc:492] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5: Bootstrap starting.
I20260810 06:45:38.352864 25360 tablet_bootstrap.cc:654] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.352921 25361 tablet_bootstrap.cc:654] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.353130 25359 tablet_bootstrap.cc:492] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe: Bootstrap starting.
I20260810 06:45:38.353762 25361 tablet_bootstrap.cc:492] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5: No bootstrap required, opened a new log
I20260810 06:45:38.353812 25361 ts_tablet_manager.cc:1397] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:38.353849 25359 tablet_bootstrap.cc:654] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.353946 25361 raft_consensus.cc:348] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.354013 25361 raft_consensus.cc:374] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.354044 25361 raft_consensus.cc:729] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4db1bb333a7141ee9e342a77941df9f5, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.354087 25361 consensus_queue.cc:260] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.354175 25361 ts_tablet_manager.cc:1428] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:38.354704 25360 tablet_bootstrap.cc:492] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f: No bootstrap required, opened a new log
I20260810 06:45:38.354768 25360 ts_tablet_manager.cc:1397] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:45:38.354907 25360 raft_consensus.cc:348] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.354990 25360 raft_consensus.cc:374] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.355015 25360 raft_consensus.cc:729] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a7072ca0a34b0ba017d9f146e6656f, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.355079 25360 consensus_queue.cc:260] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.355183 25360 ts_tablet_manager.cc:1428] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:38.355629 25359 tablet_bootstrap.cc:492] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe: No bootstrap required, opened a new log
I20260810 06:45:38.355715 25359 ts_tablet_manager.cc:1397] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 06:45:38.355897 25359 raft_consensus.cc:348] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.355983 25359 raft_consensus.cc:374] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.355998 25359 raft_consensus.cc:729] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e68488536e348e88ee3491a468288fe, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.356057 25359 consensus_queue.cc:260] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.356185 25359 ts_tablet_manager.cc:1428] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 06:45:38.479525 25419 raft_consensus.cc:482] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:38.479635 25419 raft_consensus.cc:504] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.479796 25419 leader_election.cc:288] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153), 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195:38379)
I20260810 06:45:38.480051 25108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e6e48e2a134371b0056814dbd02d41" candidate_uuid: "4db1bb333a7141ee9e342a77941df9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68488536e348e88ee3491a468288fe" is_pre_election: true
I20260810 06:45:38.480059 25276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e6e48e2a134371b0056814dbd02d41" candidate_uuid: "4db1bb333a7141ee9e342a77941df9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a7072ca0a34b0ba017d9f146e6656f" is_pre_election: true
I20260810 06:45:38.480165 25108 raft_consensus.cc:2454] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4db1bb333a7141ee9e342a77941df9f5 in term 0.
I20260810 06:45:38.480167 25276 raft_consensus.cc:2454] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4db1bb333a7141ee9e342a77941df9f5 in term 0.
I20260810 06:45:38.480350 24885 leader_election.cc:302] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4db1bb333a7141ee9e342a77941df9f5, 6e68488536e348e88ee3491a468288fe; no voters:
I20260810 06:45:38.480500 25419 raft_consensus.cc:2781] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:38.480548 25419 raft_consensus.cc:482] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:38.480600 25419 raft_consensus.cc:3037] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.481516 25419 raft_consensus.cc:504] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.481711 25419 leader_election.cc:288] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [CANDIDATE]: Term 1 election: Requested vote from peers 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153), 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195:38379)
I20260810 06:45:38.481902 25108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e6e48e2a134371b0056814dbd02d41" candidate_uuid: "4db1bb333a7141ee9e342a77941df9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68488536e348e88ee3491a468288fe"
I20260810 06:45:38.481997 25108 raft_consensus.cc:3037] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.481984 25276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e6e48e2a134371b0056814dbd02d41" candidate_uuid: "4db1bb333a7141ee9e342a77941df9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a7072ca0a34b0ba017d9f146e6656f"
I20260810 06:45:38.482066 25276 raft_consensus.cc:3037] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.482841 25108 raft_consensus.cc:2454] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4db1bb333a7141ee9e342a77941df9f5 in term 1.
I20260810 06:45:38.482846 25276 raft_consensus.cc:2454] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4db1bb333a7141ee9e342a77941df9f5 in term 1.
I20260810 06:45:38.483037 24885 leader_election.cc:302] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a7072ca0a34b0ba017d9f146e6656f, 4db1bb333a7141ee9e342a77941df9f5; no voters:
I20260810 06:45:38.483143 25419 raft_consensus.cc:2781] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:38.483197 25419 raft_consensus.cc:686] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4db1bb333a7141ee9e342a77941df9f5, State: Running, Role: LEADER
I20260810 06:45:38.483250 25419 consensus_queue.cc:237] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.483875 24786 catalog_manager.cc:5495] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4db1bb333a7141ee9e342a77941df9f5 (127.15.232.193). New cstate: current_term: 1 leader_uuid: "4db1bb333a7141ee9e342a77941df9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } health_report { overall_health: UNKNOWN } } }
I20260810 06:45:38.505383 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.503027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46110 (local address 127.15.232.193:42115)
0810 06:45:38.503092 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.503095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.503106 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:38.503137 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.503138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.503150 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.503224 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:38.503397 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.503871 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.504651 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.504844 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.504922 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.505078 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:38.505119 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:38.505207 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.505276 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.505295 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 06:45:38.506237 25276 raft_consensus.cc:1261] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Refusing update from remote peer 4db1bb333a7141ee9e342a77941df9f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:38.506289 25108 raft_consensus.cc:1261] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Refusing update from remote peer 4db1bb333a7141ee9e342a77941df9f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:38.506515 25419 consensus_queue.cc:1035] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:38.506649 25416 consensus_queue.cc:1035] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:38.509816 24786 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:38.509931 24786 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:38.510649 24786 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9: Sending DeleteTablet for 3 replicas of tablet e5e6e48e2a134371b0056814dbd02d41
I20260810 06:45:38.510901 24936 tablet_service.cc:1514] Processing DeleteTablet for tablet e5e6e48e2a134371b0056814dbd02d41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:38 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:46086
I20260810 06:45:38.510931 25087 tablet_service.cc:1514] Processing DeleteTablet for tablet e5e6e48e2a134371b0056814dbd02d41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:38 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:40164
I20260810 06:45:38.511003 25387 tablet_replica.cc:330] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5: stopping tablet replica
I20260810 06:45:38.511027 25384 tablet_replica.cc:330] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe: stopping tablet replica
I20260810 06:45:38.511008 25253 tablet_service.cc:1514] Processing DeleteTablet for tablet e5e6e48e2a134371b0056814dbd02d41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:38 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:47228
I20260810 06:45:38.511063 25387 raft_consensus.cc:2229] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:38.511072 25384 raft_consensus.cc:2229] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:38.511088 25385 tablet_replica.cc:330] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f: stopping tablet replica
I20260810 06:45:38.511118 25384 raft_consensus.cc:2258] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:38.511123 25387 raft_consensus.cc:2258] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:38.511134 25385 raft_consensus.cc:2229] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:38.511211 25385 raft_consensus.cc:2258] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:38.511628 25385 ts_tablet_manager.cc:1905] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:38.511766 25384 ts_tablet_manager.cc:1905] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:38.511780 25387 ts_tablet_manager.cc:1905] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:38.513320 24786 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:38.513494 24786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:38.513712 25385 ts_tablet_manager.cc:1918] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:38.513742 25384 ts_tablet_manager.cc:1918] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:38.513772 25385 log.cc:1196] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/e5e6e48e2a134371b0056814dbd02d41
I20260810 06:45:38.513798 25384 log.cc:1196] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/e5e6e48e2a134371b0056814dbd02d41
I20260810 06:45:38.513980 25385 ts_tablet_manager.cc:1939] T e5e6e48e2a134371b0056814dbd02d41 P 32a7072ca0a34b0ba017d9f146e6656f: Deleting consensus metadata
I20260810 06:45:38.514076 25384 ts_tablet_manager.cc:1939] T e5e6e48e2a134371b0056814dbd02d41 P 6e68488536e348e88ee3491a468288fe: Deleting consensus metadata
I20260810 06:45:38.514320 24773 catalog_manager.cc:4841] TS 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195:38379): tablet e5e6e48e2a134371b0056814dbd02d41 (table test-table [id=8b469ddd8fa04626b1dda51026c888f2]) successfully deleted
I20260810 06:45:38.514396 24773 catalog_manager.cc:4841] TS 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153): tablet e5e6e48e2a134371b0056814dbd02d41 (table test-table [id=8b469ddd8fa04626b1dda51026c888f2]) successfully deleted
I20260810 06:45:38.514624 25387 ts_tablet_manager.cc:1918] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:38.514689 25387 log.cc:1196] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-0/wal/wals/e5e6e48e2a134371b0056814dbd02d41
I20260810 06:45:38.514914 25387 ts_tablet_manager.cc:1939] T e5e6e48e2a134371b0056814dbd02d41 P 4db1bb333a7141ee9e342a77941df9f5: Deleting consensus metadata
I20260810 06:45:38.515287 25253 tablet_service.cc:1467] Processing CreateTablet for tablet d8844b7d19e44f5a94b3842c88de215f (DEFAULT_TABLE table=test-table [id=be502102db2c4568954549c2e2b17ebc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:38.515316 25087 tablet_service.cc:1467] Processing CreateTablet for tablet d8844b7d19e44f5a94b3842c88de215f (DEFAULT_TABLE table=test-table [id=be502102db2c4568954549c2e2b17ebc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:38.515384 25253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8844b7d19e44f5a94b3842c88de215f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.515422 25087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8844b7d19e44f5a94b3842c88de215f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.516192 25360 tablet_bootstrap.cc:492] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f: Bootstrap starting.
I20260810 06:45:38.515223 24774 catalog_manager.cc:4841] TS 4db1bb333a7141ee9e342a77941df9f5 (127.15.232.193:42115): tablet e5e6e48e2a134371b0056814dbd02d41 (table test-table [id=8b469ddd8fa04626b1dda51026c888f2]) successfully deleted
I20260810 06:45:38.516542 25360 tablet_bootstrap.cc:654] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.516613 24936 tablet_service.cc:1467] Processing CreateTablet for tablet d8844b7d19e44f5a94b3842c88de215f (DEFAULT_TABLE table=test-table [id=be502102db2c4568954549c2e2b17ebc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:38.516708 24936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8844b7d19e44f5a94b3842c88de215f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:38.517225 25360 tablet_bootstrap.cc:492] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f: No bootstrap required, opened a new log
I20260810 06:45:38.517256 25359 tablet_bootstrap.cc:492] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe: Bootstrap starting.
I20260810 06:45:38.517268 25360 ts_tablet_manager.cc:1397] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:38.517385 25360 raft_consensus.cc:348] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.517463 25360 raft_consensus.cc:374] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.517486 25360 raft_consensus.cc:729] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a7072ca0a34b0ba017d9f146e6656f, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.517532 25360 consensus_queue.cc:260] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.517648 25360 ts_tablet_manager.cc:1428] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:38.517727 25359 tablet_bootstrap.cc:654] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.518057 25361 tablet_bootstrap.cc:492] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5: Bootstrap starting.
I20260810 06:45:38.518352 25361 tablet_bootstrap.cc:654] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:38.518920 25359 tablet_bootstrap.cc:492] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe: No bootstrap required, opened a new log
I20260810 06:45:38.518989 25359 ts_tablet_manager.cc:1397] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 06:45:38.519047 25361 tablet_bootstrap.cc:492] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5: No bootstrap required, opened a new log
I20260810 06:45:38.519097 25361 ts_tablet_manager.cc:1397] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:38.519150 25359 raft_consensus.cc:348] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.519223 25359 raft_consensus.cc:374] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.519202 25361 raft_consensus.cc:348] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.519249 25359 raft_consensus.cc:729] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e68488536e348e88ee3491a468288fe, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.519263 25361 raft_consensus.cc:374] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:38.519282 25361 raft_consensus.cc:729] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4db1bb333a7141ee9e342a77941df9f5, State: Initialized, Role: FOLLOWER
I20260810 06:45:38.519305 25359 consensus_queue.cc:260] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.519320 25361 consensus_queue.cc:260] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.519389 25359 ts_tablet_manager.cc:1428] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 06:45:38.519397 25361 ts_tablet_manager.cc:1428] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:38.535952 25370 raft_consensus.cc:482] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:38.536084 25370 raft_consensus.cc:504] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.536295 25370 leader_election.cc:288] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4db1bb333a7141ee9e342a77941df9f5 (127.15.232.193:42115), 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153)
I20260810 06:45:38.536494 24959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8844b7d19e44f5a94b3842c88de215f" candidate_uuid: "32a7072ca0a34b0ba017d9f146e6656f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4db1bb333a7141ee9e342a77941df9f5" is_pre_election: true
I20260810 06:45:38.536521 25108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8844b7d19e44f5a94b3842c88de215f" candidate_uuid: "32a7072ca0a34b0ba017d9f146e6656f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68488536e348e88ee3491a468288fe" is_pre_election: true
I20260810 06:45:38.536605 24959 raft_consensus.cc:2454] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a7072ca0a34b0ba017d9f146e6656f in term 0.
I20260810 06:45:38.536609 25108 raft_consensus.cc:2454] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a7072ca0a34b0ba017d9f146e6656f in term 0.
I20260810 06:45:38.536779 25199 leader_election.cc:302] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a7072ca0a34b0ba017d9f146e6656f, 4db1bb333a7141ee9e342a77941df9f5; no voters:
I20260810 06:45:38.536954 25370 raft_consensus.cc:2781] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:38.537024 25370 raft_consensus.cc:482] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:38.537045 25370 raft_consensus.cc:3037] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.538125 25370 raft_consensus.cc:504] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.538276 25370 leader_election.cc:288] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [CANDIDATE]: Term 1 election: Requested vote from peers 4db1bb333a7141ee9e342a77941df9f5 (127.15.232.193:42115), 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153)
I20260810 06:45:38.538516 25108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8844b7d19e44f5a94b3842c88de215f" candidate_uuid: "32a7072ca0a34b0ba017d9f146e6656f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68488536e348e88ee3491a468288fe"
I20260810 06:45:38.538626 25108 raft_consensus.cc:3037] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.539254 25108 raft_consensus.cc:2454] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32a7072ca0a34b0ba017d9f146e6656f in term 1.
I20260810 06:45:38.539397 24959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8844b7d19e44f5a94b3842c88de215f" candidate_uuid: "32a7072ca0a34b0ba017d9f146e6656f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4db1bb333a7141ee9e342a77941df9f5"
I20260810 06:45:38.539484 24959 raft_consensus.cc:3037] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:38.540149 24959 raft_consensus.cc:2454] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32a7072ca0a34b0ba017d9f146e6656f in term 1.
I20260810 06:45:38.540290 25198 leader_election.cc:302] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a7072ca0a34b0ba017d9f146e6656f, 6e68488536e348e88ee3491a468288fe; no voters:
I20260810 06:45:38.540408 25370 raft_consensus.cc:2781] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:38.540479 25370 raft_consensus.cc:686] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 1 LEADER]: Becoming Leader. State: Replica: 32a7072ca0a34b0ba017d9f146e6656f, State: Running, Role: LEADER
I20260810 06:45:38.540534 25370 consensus_queue.cc:237] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } }
I20260810 06:45:38.541181 24785 catalog_manager.cc:5495] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f reported cstate change: term changed from 0 to 1, leader changed from <none> to 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195). New cstate: current_term: 1 leader_uuid: "32a7072ca0a34b0ba017d9f146e6656f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32a7072ca0a34b0ba017d9f146e6656f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38379 } health_report { overall_health: HEALTHY } } }
I20260810 06:45:38.549580 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.547214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.15.232.195:38379)
0810 06:45:38.547255 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.547257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.547304 (+ 47us) server_negotiation.cc:407] Connection header received
0810 06:45:38.547325 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.547325 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.547333 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.547383 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:38.547504 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.548093 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.548812 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.549078 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.549152 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.549350 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:38.549381 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:38.549424 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.549506 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.549525 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 06:45:38.550101 25330 tablet.cc:2369] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:45:38.554208 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.552176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:41201 (local address 127.15.232.193:42115)
0810 06:45:38.552248 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.552250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.552259 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:38.552283 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.552283 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.552299 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.552350 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:38.552470 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.552911 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.553683 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.553863 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.553919 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.554048 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:38.554071 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:38.554104 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.554139 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.554152 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:45:38.554229 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.552043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:42115 (local address 127.15.232.195:41201)
0810 06:45:38.552101 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.552115 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.552157 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.552364 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.552365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.552366 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:38.552424 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.552425 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.552971 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.552971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.553611 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.553614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.553932 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.553932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.553971 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.553985 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:38.554103 (+ 118us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:38.554104 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.554138 (+ 34us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.554175 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260810 06:45:38.556535 24959 raft_consensus.cc:1261] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Refusing update from remote peer 32a7072ca0a34b0ba017d9f146e6656f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:38.556548 25108 raft_consensus.cc:1261] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Refusing update from remote peer 32a7072ca0a34b0ba017d9f146e6656f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:38.556749 25366 consensus_queue.cc:1035] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4db1bb333a7141ee9e342a77941df9f5" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 42115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:38.556851 25370 consensus_queue.cc:1035] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e68488536e348e88ee3491a468288fe" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:38.557960 25373 mvcc.cc:205] Tried to move back new op lower bound from 7316866410726219776 to 7316866410662293504. Current Snapshot: MvccSnapshot[applied={T|T < 7316866410726219776}]
I20260810 06:45:38.564215 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.561887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:60003 (local address 127.15.232.195:38379)
0810 06:45:38.561928 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:38.561930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:38.561952 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:45:38.562002 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:38.562002 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:38.562014 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:38.562054 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:38.562188 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.562706 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.563423 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.563606 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.563655 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.563911 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:38.563938 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:38.564005 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:38.564099 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 06:45:38.564132 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 06:45:38.564248 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.561848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38379 (local address 127.15.232.193:60003)
0810 06:45:38.561892 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.561913 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:38.561950 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:38.562084 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:38.562085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:38.562086 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:45:38.562134 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.562136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.562782 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.562783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.563344 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:38.563346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:38.563672 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:38.563673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:38.563746 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:38.563754 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:45:38.564010 (+ 256us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:45:38.564013 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:38.564071 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 06:45:38.564130 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260810 06:45:38.570822 24786 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:38.570930 24786 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:38.571521 24786 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7553efd276804d57a18561dadc7cbba9: Sending DeleteTablet for 3 replicas of tablet d8844b7d19e44f5a94b3842c88de215f
I20260810 06:45:38.571785 25087 tablet_service.cc:1514] Processing DeleteTablet for tablet d8844b7d19e44f5a94b3842c88de215f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:38 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:40164
I20260810 06:45:38.571816 24936 tablet_service.cc:1514] Processing DeleteTablet for tablet d8844b7d19e44f5a94b3842c88de215f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:38 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:46086
I20260810 06:45:38.571821 25253 tablet_service.cc:1514] Processing DeleteTablet for tablet d8844b7d19e44f5a94b3842c88de215f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:38 UTC) from {username='oryx', principal='oryx/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:47228
I20260810 06:45:38.571911 25384 tablet_replica.cc:330] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe: stopping tablet replica
I20260810 06:45:38.571926 25385 tablet_replica.cc:330] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f: stopping tablet replica
I20260810 06:45:38.571967 25384 raft_consensus.cc:2229] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:38.571985 25385 raft_consensus.cc:2229] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:38.572010 25384 raft_consensus.cc:2258] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:38.572048 25385 raft_consensus.cc:2258] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:38.572081 25387 tablet_replica.cc:330] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5: stopping tablet replica
I20260810 06:45:38.572146 25387 raft_consensus.cc:2229] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:38.572194 25387 raft_consensus.cc:2258] T d8844b7d19e44f5a94b3842c88de215f P 4db1bb333a7141ee9e342a77941df9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:38.572403 25385 ts_tablet_manager.cc:1905] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:38.572499 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 24867
I20260810 06:45:38.572631 25384 ts_tablet_manager.cc:1905] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:38.574504 25384 ts_tablet_manager.cc:1918] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:38.574548 25384 log.cc:1196] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/d8844b7d19e44f5a94b3842c88de215f
I20260810 06:45:38.574718 25384 ts_tablet_manager.cc:1939] T d8844b7d19e44f5a94b3842c88de215f P 6e68488536e348e88ee3491a468288fe: Deleting consensus metadata
I20260810 06:45:38.575591 24773 catalog_manager.cc:4841] TS 6e68488536e348e88ee3491a468288fe (127.15.232.194:39153): tablet d8844b7d19e44f5a94b3842c88de215f (table test-table [id=be502102db2c4568954549c2e2b17ebc]) successfully deleted
I20260810 06:45:38.580367 25385 ts_tablet_manager.cc:1918] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:45:38.580482 25385 log.cc:1196] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/d8844b7d19e44f5a94b3842c88de215f
I20260810 06:45:38.580756 25385 ts_tablet_manager.cc:1939] T d8844b7d19e44f5a94b3842c88de215f P 32a7072ca0a34b0ba017d9f146e6656f: Deleting consensus metadata
I20260810 06:45:38.581182 24773 catalog_manager.cc:4841] TS 32a7072ca0a34b0ba017d9f146e6656f (127.15.232.195:38379): tablet d8844b7d19e44f5a94b3842c88de215f (table test-table [id=be502102db2c4568954549c2e2b17ebc]) successfully deleted
W20260810 06:45:38.583431 24774 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:42115 (error 108)
I20260810 06:45:38.583698 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 25018
I20260810 06:45:38.586540 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:38.586375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:42115 (local address 127.0.0.1:46114)
0810 06:45:38.586443 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:38.586507 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:42115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:45:38.586685 24774 catalog_manager.cc:4584] TS 4db1bb333a7141ee9e342a77941df9f5 (127.15.232.193:42115): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8844b7d19e44f5a94b3842c88de215f: Network error: Client connection negotiation failed: client connection to 127.15.232.193:42115: connect: Connection refused (error 111)
I20260810 06:45:38.591698 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 25177
I20260810 06:45:38.600098 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 24753
2026-08-10T06:45:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2917 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[25456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[25456](info): set up 2 sockets
Aug 10 06:45:38 dist-test-slave-vxj2 krb5kdc[25456](info): commencing operation
krb5kdc: starting...
W20260810 06:45:39.738359 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:39 dist-test-slave-vxj2 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:45:39.865123 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:38531
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:38531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:39.952168 25470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:39.952380 25470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:39.952404 25470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:39.953511 25470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:39.953545 25470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:39.953559 25470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:39.953572 25470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:39.953583 25470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:39.955035 25470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:38531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:38531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:39.955240 25470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:39.955403 25470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:39.960397 25479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:39.960403 25476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:39.960582 25470 server_base.cc:1034] running on GCE node
W20260810 06:45:39.960531 25477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:39.960947 25470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:39.961306 25470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:39.962525 25470 hybrid_clock.cc:648] HybridClock initialized: now 1786344339962516 us; error 40 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T06:45:39Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1370 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:40 dist-test-slave-vxj2 krb5kdc[25485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:40 dist-test-slave-vxj2 krb5kdc[25485](info): set up 2 sockets
Aug 10 06:45:40 dist-test-slave-vxj2 krb5kdc[25485](info): commencing operation
krb5kdc: starting...
W20260810 06:45:41.094954 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:45:41 dist-test-slave-vxj2 krb5kdc[25485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:45:41.225243 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:34471
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:34471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:41.313390 25501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:41.313634 25501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:41.313661 25501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:41.314939 25501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:41.314980 25501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:41.314996 25501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:41.315011 25501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:41.315024 25501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:41.316602 25501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:34471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:34471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:41.316825 25501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:41.317025 25501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:41.322443 25510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:41.322408 25508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:41.322551 25501 server_base.cc:1034] running on GCE node
W20260810 06:45:41.322418 25507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:41.323020 25501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:41.323403 25501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:41.324597 25501 hybrid_clock.cc:648] HybridClock initialized: now 1786344341324581 us; error 50 us; skew 500 ppm
Aug 10 06:45:41 dist-test-slave-vxj2 krb5kdc[25485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344341, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:41.332746 25501 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:45:41.333292 25501 webserver.cc:460] Webserver started at http://127.15.232.254:33783/ using document root <none> and password file <none>
I20260810 06:45:41.333539 25501 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:41.333609 25501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:41.333760 25501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:41.334867 25501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "4bd03e0ee80740c098c53a8ddfbe39fa"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "31ec5838d62426be83eafebcf1d8f17e"
server_key_iv: "f67f11ae371ef5327ad9480ce90277a5"
server_key_version: "encryptionkey@0"
I20260810 06:45:41.335246 25501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "4bd03e0ee80740c098c53a8ddfbe39fa"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "31ec5838d62426be83eafebcf1d8f17e"
server_key_iv: "f67f11ae371ef5327ad9480ce90277a5"
server_key_version: "encryptionkey@0"
I20260810 06:45:41.336650 25501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:45:41.337512 25519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:41.337694 25501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:41.337809 25501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "4bd03e0ee80740c098c53a8ddfbe39fa"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "31ec5838d62426be83eafebcf1d8f17e"
server_key_iv: "f67f11ae371ef5327ad9480ce90277a5"
server_key_version: "encryptionkey@0"
I20260810 06:45:41.337886 25501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:41.351604 25501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:41.354372 25501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:41.354562 25501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:41.358748 25501 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:34471
I20260810 06:45:41.358793 25575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:34471 every 8 connection(s)
I20260810 06:45:41.359189 25501 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:41.360059 25576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:41.360420 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 25501
I20260810 06:45:41.360536 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:41.360751 16291 external_mini_cluster.cc:1428] Setting key 1bc67212fc0e0c94a9c0d496dbf2db54
I20260810 06:45:41.363344 25576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa: Bootstrap starting.
I20260810 06:45:41.363969 25576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:41.364308 25576 log.cc:824] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:41.365099 25576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa: No bootstrap required, opened a new log
Aug 10 06:45:41 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:41.367749 25576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34471 } }
I20260810 06:45:41.367940 25576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:41.368018 25576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bd03e0ee80740c098c53a8ddfbe39fa, State: Initialized, Role: FOLLOWER
I20260810 06:45:41.368160 25576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34471 } }
I20260810 06:45:41.368225 25576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:41.368247 25576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:41.368319 25576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:41.368876 25576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34471 } }
I20260810 06:45:41.368965 25576 leader_election.cc:302] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bd03e0ee80740c098c53a8ddfbe39fa; no voters:
I20260810 06:45:41.369184 25576 leader_election.cc:288] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:41.369213 25582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:41.369333 25582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 1 LEADER]: Becoming Leader. State: Replica: 4bd03e0ee80740c098c53a8ddfbe39fa, State: Running, Role: LEADER
I20260810 06:45:41.369421 25582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34471 } }
I20260810 06:45:41.369485 25576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:41.369773 25582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34471 } } }
I20260810 06:45:41.369851 25582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:41.369975 25582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bd03e0ee80740c098c53a8ddfbe39fa. Latest consensus state: current_term: 1 leader_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34471 } } }
I20260810 06:45:41.370070 25582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [sys.catalog]: This master's current role is: LEADER
W20260810 06:45:41.371279 25598 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:45:41.371335 25598 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:45:41.371624 25587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:41.371801 25587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:41.372079 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:41.361370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48608 (local address 127.15.232.254:34471)
0810 06:45:41.361563 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:41.361574 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:41.361606 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:45:41.362027 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:41.362034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:41.362080 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:41.362199 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:41.363023 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.363965 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.365028 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.365492 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.365576 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:41.368406 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:41.368411 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:41.368411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:41.368430 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:41.370430 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:41.370797 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:41.370800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:41.370800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:41.370915 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:41.371139 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:41.371140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:41.371140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:41.371426 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:41.371612 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:41.371723 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 06:45:41.371867 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260810 06:45:41.373373 25587 catalog_manager.cc:1347] Generated new cluster ID: 7d1ba2c7b8e74403b670ba2d46d556e2
I20260810 06:45:41.373430 25587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:41.379976 25587 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:41.380736 25587 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:41.389051 25587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa: Generated new TSK 0
I20260810 06:45:41.389299 25587 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:45:41.420253 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:34471
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:41.507604 25607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:41.507822 25607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:41.507848 25607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:41.509234 25607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:41.509279 25607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:41.509305 25607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:41.510787 25607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:34471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:41.511063 25607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:41.511302 25607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:41.516508 25616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:41.516510 25614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:41.516602 25613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:41.516742 25607 server_base.cc:1034] running on GCE node
I20260810 06:45:41.517031 25607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:41.517305 25607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:41.518481 25607 hybrid_clock.cc:648] HybridClock initialized: now 1786344341518471 us; error 50 us; skew 500 ppm
Aug 10 06:45:41 dist-test-slave-vxj2 krb5kdc[25485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344341, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:41.521045 25607 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:45:41.521515 25607 webserver.cc:460] Webserver started at http://127.15.232.193:34491/ using document root <none> and password file <none>
I20260810 06:45:41.521720 25607 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:41.521760 25607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:41.521852 25607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:41.522879 25607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "dcdd741581ed40a5ab175a8c5996ac35"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "4672077944f51cb9d3bf51a2b78578a0"
server_key_iv: "325b58ec25ad5de97f58ba86b031c9f4"
server_key_version: "encryptionkey@0"
I20260810 06:45:41.523216 25607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "dcdd741581ed40a5ab175a8c5996ac35"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "4672077944f51cb9d3bf51a2b78578a0"
server_key_iv: "325b58ec25ad5de97f58ba86b031c9f4"
server_key_version: "encryptionkey@0"
I20260810 06:45:41.524551 25607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:41.525368 25623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:41.525571 25607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:41.525667 25607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "dcdd741581ed40a5ab175a8c5996ac35"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "4672077944f51cb9d3bf51a2b78578a0"
server_key_iv: "325b58ec25ad5de97f58ba86b031c9f4"
server_key_version: "encryptionkey@0"
I20260810 06:45:41.525746 25607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:41.536785 25607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:41.539191 25607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:41.539315 25607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:41.539512 25607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:41.539856 25607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:41.539899 25607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:41.539923 25607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:41.539969 25607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:41.546342 25607 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:43017
I20260810 06:45:41.546404 25756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:43017 every 8 connection(s)
I20260810 06:45:41.546792 25607 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
Aug 10 06:45:41 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344341, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:41.556526 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 25607
I20260810 06:45:41.556663 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:41.556893 16291 external_mini_cluster.cc:1428] Setting key 6c582d536edf3693f9957b889daf528a
I20260810 06:45:41.557678 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:41.547604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:38065 (local address 127.15.232.254:34471)
0810 06:45:41.547656 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:41.547658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:41.547929 (+ 271us) server_negotiation.cc:407] Connection header received
0810 06:45:41.548528 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:41.548530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:41.548544 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:41.548600 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:41.549777 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.550327 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.551461 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.551694 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.551797 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:41.554272 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:41.554273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:41.554273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:41.554281 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:41.555773 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:41.556203 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:41.556204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:41.556205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:41.556294 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:41.556667 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:41.556669 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:41.556669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:41.556900 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:41.557006 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:41.557455 (+ 449us) server_negotiation.cc:299] Negotiation successful
0810 06:45:41.557557 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:45:41.557847 25759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:41.547569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:38065)
0810 06:45:41.547777 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:41.547817 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:41.548405 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:41.548702 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:41.548706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:41.549412 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:41.549645 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:41.549652 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.550440 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.550442 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:41.551354 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:41.551356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.551811 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.551812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:41.551953 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:41.552548 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:41.552558 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:41.554156 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:41.555874 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:41.555880 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:41.555882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:41.556117 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:41.556370 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:41.556371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:41.556371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:41.556536 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:41.557045 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:41.557053 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:41.557294 (+ 241us) client_negotiation.cc:769] Sending connection context
0810 06:45:41.557412 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 06:45:41.557551 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
I20260810 06:45:41.558439 25757 heartbeater.cc:347] Connected to a master server at 127.15.232.254:34471
I20260810 06:45:41.558558 25757 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:41.558770 25757 heartbeater.cc:511] Master 127.15.232.254:34471 requested a full tablet report, sending...
I20260810 06:45:41.559307 25537 ts_manager.cc:194] Registered new tserver with Master: dcdd741581ed40a5ab175a8c5996ac35 (127.15.232.193:43017)
I20260810 06:45:41.559964 25537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:38065
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:45:41.586512 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:34471
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:41.678226 25762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:41.678442 25762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:41.678467 25762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:41.679665 25762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:41.679702 25762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:41.679730 25762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:41.681236 25762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:34471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:41.681437 25762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:41.681690 25762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:41.687139 25768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:41.687189 25769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:41.687494 25772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:41.687440 25762 server_base.cc:1034] running on GCE node
I20260810 06:45:41.687914 25762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:41.688223 25762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:41.689459 25762 hybrid_clock.cc:648] HybridClock initialized: now 1786344341689434 us; error 38 us; skew 500 ppm
Aug 10 06:45:41 dist-test-slave-vxj2 krb5kdc[25485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344341, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:41.692245 25762 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:45:41.692737 25762 webserver.cc:460] Webserver started at http://127.15.232.194:34163/ using document root <none> and password file <none>
I20260810 06:45:41.693004 25762 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:41.693080 25762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:41.693288 25762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:41.694483 25762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "26aaa4834de3451d821ee7c426a935f6"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "4c4759aa6d1a3aeb7f9330ae5d8382e9"
server_key_iv: "bcb3ae20dbd4be28dd1de8c356d31786"
server_key_version: "encryptionkey@0"
I20260810 06:45:41.694882 25762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "26aaa4834de3451d821ee7c426a935f6"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "4c4759aa6d1a3aeb7f9330ae5d8382e9"
server_key_iv: "bcb3ae20dbd4be28dd1de8c356d31786"
server_key_version: "encryptionkey@0"
I20260810 06:45:41.696442 25762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:41.697335 25780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:41.697533 25762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:41.697654 25762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "26aaa4834de3451d821ee7c426a935f6"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "4c4759aa6d1a3aeb7f9330ae5d8382e9"
server_key_iv: "bcb3ae20dbd4be28dd1de8c356d31786"
server_key_version: "encryptionkey@0"
I20260810 06:45:41.697745 25762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:41.713425 25762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:41.715679 25762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:41.715865 25762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:41.716136 25762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:41.716487 25762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:41.716550 25762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:41.716598 25762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:41.716625 25762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:41.723331 25762 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:36901
I20260810 06:45:41.723376 25908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:36901 every 8 connection(s)
I20260810 06:45:41.723762 25762 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:45:41 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344341, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:41.732466 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 25762
I20260810 06:45:41.732560 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:41.732824 16291 external_mini_cluster.cc:1428] Setting key 666d7380473010c155b91a8477a9a8c3
I20260810 06:45:41.734716 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:41.724656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:52833 (local address 127.15.232.254:34471)
0810 06:45:41.724771 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:41.724773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:41.724962 (+ 189us) server_negotiation.cc:407] Connection header received
0810 06:45:41.725622 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:41.725624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:41.725638 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:41.725712 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:41.726737 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.727260 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.728354 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.728622 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.728737 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:41.731377 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:41.731378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:41.731379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:41.731388 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:41.733128 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:41.733601 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:41.733603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:41.733603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:41.733671 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:41.733981 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:41.733982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:41.733982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:41.734106 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:41.734192 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:41.734510 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 06:45:41.734614 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 06:45:41.734959 25913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:41.724564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:52833)
0810 06:45:41.724811 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:41.724861 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:41.725488 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:41.725779 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:41.725784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:41.726399 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:41.726616 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:41.726621 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.727394 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.727396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:41.728246 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:41.728249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.728748 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.728750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:41.728890 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:41.729544 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:41.729554 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:41.731260 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:41.733250 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:41.733258 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:41.733261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:41.733510 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:41.733768 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:41.733769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:41.733769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:41.733895 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:41.734183 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:41.734190 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:41.734361 (+ 171us) client_negotiation.cc:769] Sending connection context
0810 06:45:41.734487 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 06:45:41.734676 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":105,"threads_started":1}
I20260810 06:45:41.735469 25910 heartbeater.cc:347] Connected to a master server at 127.15.232.254:34471
I20260810 06:45:41.735585 25910 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:41.735764 25910 heartbeater.cc:511] Master 127.15.232.254:34471 requested a full tablet report, sending...
I20260810 06:45:41.736178 25537 ts_manager.cc:194] Registered new tserver with Master: 26aaa4834de3451d821ee7c426a935f6 (127.15.232.194:36901)
I20260810 06:45:41.736568 25537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:52833
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:45:41.760279 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:34471
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:41.849458 25916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:41.849745 25916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:41.849790 25916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:41.851089 25916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:41.851173 25916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:41.851225 25916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:41.852903 25916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:34471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:41.853235 25916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:41.853508 25916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:41.859149 25924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:41.859150 25922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:41.859156 25921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:41.859506 25916 server_base.cc:1034] running on GCE node
I20260810 06:45:41.859711 25916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:41.859934 25916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:41.861128 25916 hybrid_clock.cc:648] HybridClock initialized: now 1786344341861100 us; error 43 us; skew 500 ppm
Aug 10 06:45:41 dist-test-slave-vxj2 krb5kdc[25485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344341, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:41.863991 25916 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:45:41.864490 25916 webserver.cc:460] Webserver started at http://127.15.232.195:43125/ using document root <none> and password file <none>
I20260810 06:45:41.864740 25916 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:41.864816 25916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:41.864979 25916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:41.866171 25916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "77c0602916614ecd97b1fd30070c8c80"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "7f84bdd054ef3025577bb311c65b9556"
server_key_iv: "1dc7cbb54e3dfe2021539ad09ec00175"
server_key_version: "encryptionkey@0"
I20260810 06:45:41.866600 25916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "77c0602916614ecd97b1fd30070c8c80"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "7f84bdd054ef3025577bb311c65b9556"
server_key_iv: "1dc7cbb54e3dfe2021539ad09ec00175"
server_key_version: "encryptionkey@0"
I20260810 06:45:41.868096 25916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:41.868969 25931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:41.869213 25916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:41.869313 25916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "77c0602916614ecd97b1fd30070c8c80"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "7f84bdd054ef3025577bb311c65b9556"
server_key_iv: "1dc7cbb54e3dfe2021539ad09ec00175"
server_key_version: "encryptionkey@0"
I20260810 06:45:41.869419 25916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:41.885583 25916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:41.887655 25916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:41.887825 25916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:41.888077 25916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:41.888453 25916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:41.888512 25916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:41.888557 25916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:41.888586 25916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:41.894996 25916 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:38079
I20260810 06:45:41.895038 26057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:38079 every 8 connection(s)
I20260810 06:45:41.895426 25916 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:41.895653 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 25916
I20260810 06:45:41.895754 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:41.895984 16291 external_mini_cluster.cc:1428] Setting key 55ae97fa7ec51a0f7d51993bec71bf7c
Aug 10 06:45:41 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344341, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:45:41.905690 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:41.896285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:46295 (local address 127.15.232.254:34471)
0810 06:45:41.896346 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:41.896348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:41.896594 (+ 246us) server_negotiation.cc:407] Connection header received
0810 06:45:41.897197 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:41.897198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:41.897211 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:41.897267 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:41.898309 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.898838 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.899869 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.900074 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.900152 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:41.902717 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:41.902721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:41.902721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:41.902734 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:41.904176 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:41.904609 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:41.904610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:41.904610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:41.904675 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:41.904975 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:41.904976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:41.904977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:41.905103 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:41.905204 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:41.905503 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 06:45:41.905592 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:45:41.905908 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:41.896261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:46295)
0810 06:45:41.896447 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:41.896491 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:41.897044 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:41.897362 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:41.897366 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:41.897993 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:41.898199 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:41.898205 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.898957 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.898958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:41.899764 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:41.899767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:41.900160 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:41.900161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:41.900321 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:41.900938 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:41.900947 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:41.902606 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:41.904280 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:41.904289 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:41.904291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:41.904529 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:41.904752 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:41.904753 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:41.904753 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:41.904871 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:41.905199 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:41.905207 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:41.905359 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 06:45:41.905481 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 06:45:41.905652 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260810 06:45:41.906669 26058 heartbeater.cc:347] Connected to a master server at 127.15.232.254:34471
I20260810 06:45:41.906769 26058 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:41.906972 26058 heartbeater.cc:511] Master 127.15.232.254:34471 requested a full tablet report, sending...
I20260810 06:45:41.907470 25537 ts_manager.cc:194] Registered new tserver with Master: 77c0602916614ecd97b1fd30070c8c80 (127.15.232.195:38079)
I20260810 06:45:41.907974 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:45:41.907999 25537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:46295
WARNING: no policy specified for oryx/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab.
Entry for principal oryx/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab.
WARNING: no policy specified for oryx/127.15.232.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab.
Entry for principal oryx/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab.
WARNING: no policy specified for oryx/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab.
Entry for principal oryx/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab.
I20260810 06:45:41.991705 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 25607
I20260810 06:45:42.002096 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 25762
I20260810 06:45:42.010637 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 25916
I20260810 06:45:42.017644 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 25501
I20260810 06:45:42.024896 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:34471
--webserver_interface=127.15.232.254
--webserver_port=33783
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:34471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:42.117947 26069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:42.118201 26069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:42.118237 26069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:42.119578 26069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:42.119639 26069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:42.119657 26069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:42.119709 26069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:42.119747 26069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:42.121450 26069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:34471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:34471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=33783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:42.121758 26069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:42.122038 26069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:45:42.127614 26069 server_base.cc:1034] running on GCE node
W20260810 06:45:42.127584 26078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:42.127583 26076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:42.127622 26075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:42.128180 26069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:42.128475 26069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:42.129669 26069 hybrid_clock.cc:648] HybridClock initialized: now 1786344342129642 us; error 44 us; skew 500 ppm
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344342, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:42.132577 26069 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:45:42.133251 26069 webserver.cc:460] Webserver started at http://127.15.232.254:33783/ using document root <none> and password file <none>
I20260810 06:45:42.133512 26069 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:42.133605 26069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:42.135241 26069 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:45:42.135972 26086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:42.136178 26069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:42.136296 26069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "4bd03e0ee80740c098c53a8ddfbe39fa"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "31ec5838d62426be83eafebcf1d8f17e"
server_key_iv: "f67f11ae371ef5327ad9480ce90277a5"
server_key_version: "encryptionkey@0"
I20260810 06:45:42.136678 26069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:42.163234 26069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:42.165510 26069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:42.165715 26069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:42.169976 26069 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:34471
I20260810 06:45:42.170017 26145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:34471 every 8 connection(s)
I20260810 06:45:42.170409 26069 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:42.171988 26146 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:45:42.172640 26146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa: Bootstrap starting.
I20260810 06:45:42.180598 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 26069
I20260810 06:45:42.181031 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:43017
--local_ip_for_outbound_sockets=127.15.232.193
--tserver_master_addrs=127.15.232.254:34471
--webserver_port=34491
--webserver_interface=127.15.232.193
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.232.193
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:45:42.181666 26146 log.cc:824] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:42.183641 26146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:45:42.183974 26146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa: Bootstrap complete.
I20260810 06:45:42.186590 26146 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34471 } }
I20260810 06:45:42.186831 26146 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bd03e0ee80740c098c53a8ddfbe39fa, State: Initialized, Role: FOLLOWER
I20260810 06:45:42.187012 26146 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34471 } }
I20260810 06:45:42.187093 26146 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:42.187148 26146 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:42.187202 26146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:45:42.188305 26146 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34471 } }
I20260810 06:45:42.188401 26146 leader_election.cc:302] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bd03e0ee80740c098c53a8ddfbe39fa; no voters:
I20260810 06:45:42.188642 26146 leader_election.cc:288] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:45:42.188681 26151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:45:42.188827 26151 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [term 2 LEADER]: Becoming Leader. State: Replica: 4bd03e0ee80740c098c53a8ddfbe39fa, State: Running, Role: LEADER
I20260810 06:45:42.188948 26146 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:42.188922 26151 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34471 } }
I20260810 06:45:42.189302 26153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bd03e0ee80740c098c53a8ddfbe39fa. Latest consensus state: current_term: 2 leader_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34471 } } }
I20260810 06:45:42.189306 26152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd03e0ee80740c098c53a8ddfbe39fa" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34471 } } }
I20260810 06:45:42.189400 26153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:42.189465 26152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:42.189669 26157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:42.190254 26157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:42.190521 26157 catalog_manager.cc:1259] Loaded cluster ID: 7d1ba2c7b8e74403b670ba2d46d556e2
I20260810 06:45:42.190639 26157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:42.191831 26157 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:42.192030 26157 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4bd03e0ee80740c098c53a8ddfbe39fa: Loaded TSK: 0
I20260810 06:45:42.192313 26157 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:45:42.282635 26149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:42.282902 26149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:42.282950 26149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:42.284173 26149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:42.284245 26149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:42.284292 26149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:42.285873 26149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.193.keytab
--principal=oryx/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:43017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=34491
--tserver_master_addrs=127.15.232.254:34471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:42.286188 26149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:42.286473 26149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:42.291944 26174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:42.291994 26177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:42.292016 26175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:42.292558 26149 server_base.cc:1034] running on GCE node
I20260810 06:45:42.292791 26149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:42.293066 26149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:42.294255 26149 hybrid_clock.cc:648] HybridClock initialized: now 1786344342294252 us; error 41 us; skew 500 ppm
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344342, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:42.297344 26149 init.cc:377] Logged in from keytab as oryx/127.15.232.193@KRBTEST.COM (short username oryx)
I20260810 06:45:42.297847 26149 webserver.cc:460] Webserver started at http://127.15.232.193:34491/ using document root <none> and password file <none>
I20260810 06:45:42.298100 26149 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:42.298190 26149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:42.299674 26149 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:42.300343 26184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:42.300535 26149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:42.300627 26149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "dcdd741581ed40a5ab175a8c5996ac35"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "4672077944f51cb9d3bf51a2b78578a0"
server_key_iv: "325b58ec25ad5de97f58ba86b031c9f4"
server_key_version: "encryptionkey@0"
I20260810 06:45:42.300957 26149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:42.312942 26149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:42.315326 26149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:42.315474 26149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:42.315727 26149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:42.316042 26149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:42.316076 26149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:42.316098 26149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:42.316134 26149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:42.322731 26149 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:43017
I20260810 06:45:42.322785 26313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:43017 every 8 connection(s)
I20260810 06:45:42.323128 26149 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:42.326838 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 26149
I20260810 06:45:42.327157 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:36901
--local_ip_for_outbound_sockets=127.15.232.194
--tserver_master_addrs=127.15.232.254:34471
--webserver_port=34163
--webserver_interface=127.15.232.194
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.232.194
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:42.332801 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:42.323870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:44567)
0810 06:45:42.324057 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:42.324101 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:42.324602 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:42.325089 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:42.325092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:42.325732 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:42.325922 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.325927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.327652 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.327654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.328894 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.328897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.329587 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.329588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.329724 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.330270 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:42.330279 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:42.332582 (+ 2303us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":109,"threads_started":1}
W20260810 06:45:42.333244 26314 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:34471 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.232.254:34471: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
W20260810 06:45:42.333282 26319 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:42.323940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:44567 (local address 127.15.232.254:34471)
0810 06:45:42.324134 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:42.324144 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:42.324184 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:45:42.324824 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:42.324831 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:42.324891 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:42.325054 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:42.326058 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.327541 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.328996 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.329485 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.329607 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.332716 (+ 3109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:44567: BlockingRecv error: recv got EOF from 127.15.232.193:44567 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:42.337849 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:42.333491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:36051)
0810 06:45:42.333591 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:42.333600 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:42.333636 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:42.333821 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:42.333822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:42.333965 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:42.334008 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.334010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.334678 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.334679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.335178 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.335180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.335537 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.335538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.335609 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.336033 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:42.336038 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:42.337767 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84}
W20260810 06:45:42.337905 26319 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:42.333590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:36051 (local address 127.15.232.254:34471)
0810 06:45:42.333709 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:42.333712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:42.333718 (+ 6us) server_negotiation.cc:407] Connection header received
0810 06:45:42.333735 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:42.333736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:42.333746 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:42.333817 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:42.334088 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.334592 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.335265 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.335465 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.335540 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.337842 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:36051: BlockingRecv error: recv got EOF from 127.15.232.193:36051 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":81,"threads_started":1}
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:42.342697 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:42.338354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:34295)
0810 06:45:42.338396 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:42.338405 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:42.338438 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:42.338581 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:42.338582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:42.338713 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:42.338737 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.338738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.339349 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.339350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.339922 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.339924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.340294 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.340295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.340347 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.340880 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:42.340886 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:42.342618 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 06:45:42.342746 26319 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:42.338414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:34295 (local address 127.15.232.254:34471)
0810 06:45:42.338463 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:42.338465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:42.338474 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:42.338495 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:42.338496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:42.338506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:42.338578 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:42.338820 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.339269 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.340026 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.340211 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.340283 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.342685 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:34295: BlockingRecv error: recv got EOF from 127.15.232.193:34295 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260810 06:45:42.343051 26314 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:45:42.419620 26320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:42.419860 26320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:42.419886 26320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:42.421020 26320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:42.421061 26320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:42.421087 26320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:42.422529 26320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.194.keytab
--principal=oryx/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:36901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=34163
--tserver_master_addrs=127.15.232.254:34471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:42.422727 26320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:42.422931 26320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:42.428278 26331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:42.428385 26320 server_base.cc:1034] running on GCE node
W20260810 06:45:42.428320 26328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:42.428278 26327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:42.428750 26320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:42.429013 26320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:42.430183 26320 hybrid_clock.cc:648] HybridClock initialized: now 1786344342430177 us; error 40 us; skew 500 ppm
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344342, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:42.432821 26320 init.cc:377] Logged in from keytab as oryx/127.15.232.194@KRBTEST.COM (short username oryx)
I20260810 06:45:42.433342 26320 webserver.cc:460] Webserver started at http://127.15.232.194:34163/ using document root <none> and password file <none>
I20260810 06:45:42.433579 26320 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:42.433666 26320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:42.435071 26320 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:42.435670 26338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:42.435849 26320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:45:42.435951 26320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "26aaa4834de3451d821ee7c426a935f6"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "4c4759aa6d1a3aeb7f9330ae5d8382e9"
server_key_iv: "bcb3ae20dbd4be28dd1de8c356d31786"
server_key_version: "encryptionkey@0"
I20260810 06:45:42.436331 26320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:42.461422 26320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:42.463474 26320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:42.463644 26320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:42.463917 26320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:42.464275 26320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:42.464339 26320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:42.464385 26320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:42.464421 26320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:42.470454 26320 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:36901
I20260810 06:45:42.470502 26471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:36901 every 8 connection(s)
I20260810 06:45:42.470846 26320 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:42.473371 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 26320
I20260810 06:45:42.473711 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:38079
--local_ip_for_outbound_sockets=127.15.232.195
--tserver_master_addrs=127.15.232.254:34471
--webserver_port=43125
--webserver_interface=127.15.232.195
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.232.195
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:45:42.478927 26319 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:42.471600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:44393 (local address 127.15.232.254:34471)
0810 06:45:42.471675 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:42.471678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:42.471958 (+ 280us) server_negotiation.cc:407] Connection header received
0810 06:45:42.472446 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:42.472447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:42.472461 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:42.472509 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:42.473529 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.474290 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.475274 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.475475 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.475565 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.478870 (+ 3305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:44393: BlockingRecv error: recv got EOF from 127.15.232.194:44393 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:45:42.478992 26474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:42.471601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:44393)
0810 06:45:42.471806 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:42.471858 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:42.472354 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:42.472572 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:42.472576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:42.473206 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:42.473423 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.473428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.474381 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.474382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.475177 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.475179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.475578 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.475579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.475700 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.476253 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:42.476261 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:42.478787 (+ 2526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":94,"threads_started":1}
W20260810 06:45:42.479195 26472 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:34471 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.232.254:34471: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:42.483800 26474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:42.479426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:50453)
0810 06:45:42.479478 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:42.479489 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:42.479548 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:42.479684 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:42.479685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:42.479822 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:42.479868 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.479869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.480499 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.480500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.481001 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.481002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.481426 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.481427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.481498 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.481966 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:42.481971 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:42.483703 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 06:45:42.483843 26319 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:42.479505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:50453 (local address 127.15.232.254:34471)
0810 06:45:42.479574 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:42.479576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:42.479585 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:42.479610 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:42.479611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:42.479620 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:42.479673 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:42.479946 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.480417 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.481092 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.481353 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.481427 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.483799 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:50453: BlockingRecv error: recv got EOF from 127.15.232.194:50453 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:42.488762 26474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:42.484219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:55773)
0810 06:45:42.484303 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:42.484313 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:42.484372 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:42.484521 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:42.484522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:42.484647 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:42.484671 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.484672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.485376 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.485377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.485860 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.485862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.486235 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.486235 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.486298 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.486749 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:42.486754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:42.488682 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 06:45:42.488816 26319 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:42.484305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:55773 (local address 127.15.232.254:34471)
0810 06:45:42.484363 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:42.484366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:42.484374 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:42.484439 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:42.484440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:42.484452 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:42.484519 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:42.484745 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.485289 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.485955 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.486135 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.486235 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.488754 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:55773: BlockingRecv error: recv got EOF from 127.15.232.194:55773 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 06:45:42.489248 26472 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:45:42.559056 26476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:42.559259 26476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:42.559283 26476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:42.560479 26476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:42.560525 26476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:42.560554 26476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:42.562057 26476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/krb5kdc/oryx_127.15.232.195.keytab
--principal=oryx/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:38079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=43125
--tserver_master_addrs=127.15.232.254:34471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:42.562321 26476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:42.562575 26476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:42.567901 26485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:42.567950 26483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:42.567914 26482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:42.568220 26476 server_base.cc:1034] running on GCE node
I20260810 06:45:42.568421 26476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:42.568679 26476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:42.569821 26476 hybrid_clock.cc:648] HybridClock initialized: now 1786344342569809 us; error 27 us; skew 500 ppm
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344342, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:42.572495 26476 init.cc:377] Logged in from keytab as oryx/127.15.232.195@KRBTEST.COM (short username oryx)
I20260810 06:45:42.572996 26476 webserver.cc:460] Webserver started at http://127.15.232.195:43125/ using document root <none> and password file <none>
I20260810 06:45:42.573311 26476 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:42.573411 26476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:42.574826 26476 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:42.575513 26495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:42.575693 26476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:42.575796 26476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "77c0602916614ecd97b1fd30070c8c80"
format_stamp: "Formatted at 2026-08-10 06:45:41 on dist-test-slave-vxj2"
server_key: "7f84bdd054ef3025577bb311c65b9556"
server_key_iv: "1dc7cbb54e3dfe2021539ad09ec00175"
server_key_version: "encryptionkey@0"
I20260810 06:45:42.576159 26476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:42.588436 26476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:42.590996 26476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:42.591174 26476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:42.591451 26476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:42.591784 26476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:42.591840 26476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:42.591876 26476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:42.591904 26476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:42.598228 26476 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:38079
I20260810 06:45:42.598279 26619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:38079 every 8 connection(s)
I20260810 06:45:42.598630 26476 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:42.599503 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 26476
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:45:42.607263 26319 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:42.599336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:45565 (local address 127.15.232.254:34471)
0810 06:45:42.599388 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:42.599391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:42.599647 (+ 256us) server_negotiation.cc:407] Connection header received
0810 06:45:42.600458 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:42.600459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:42.600472 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:42.600527 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:42.601689 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.602354 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.603650 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.603876 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.603932 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.607182 (+ 3250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:45565: BlockingRecv error: recv got EOF from 127.15.232.195:45565 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:45:42.607277 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:42.599326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:45565)
0810 06:45:42.599505 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:42.599541 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:42.600351 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:42.600607 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:42.600610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:42.601317 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:42.601592 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.601599 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.602457 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.602458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.603565 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.603568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.603970 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.603971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.604139 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.604605 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:42.604616 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:42.607085 (+ 2469us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
W20260810 06:45:42.607522 26620 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:34471 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.232.254:34471: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
I20260810 06:45:42.608325 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:42.600112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48616 (local address 127.15.232.254:34471)
0810 06:45:42.600202 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:42.600204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:42.600257 (+ 53us) server_negotiation.cc:407] Connection header received
0810 06:45:42.600322 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:42.600324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:42.600337 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:42.600423 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:42.601077 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.601847 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.602829 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.603145 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.603223 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.604748 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:42.604752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:42.604752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:42.604762 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:42.606597 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:42.607035 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:42.607037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:42.607038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:42.607188 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:42.607446 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:42.607447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:42.607447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:42.607780 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:42.607929 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:42.608101 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 06:45:42.608193 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:42.612293 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:42.607750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:40731)
0810 06:45:42.607802 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:42.607813 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:42.607856 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:42.607998 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:42.607999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:42.608149 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:42.608202 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.608204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.609040 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.609041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.609619 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.609621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.609998 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.609999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.610074 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.610496 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:42.610502 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:42.612246 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:45:42.612412 26319 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:42.607814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:40731 (local address 127.15.232.254:34471)
0810 06:45:42.607855 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:42.607857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:42.607865 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:42.607914 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:42.607914 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:42.607924 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:42.608009 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:42.608282 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.608962 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.609705 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.609912 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.609987 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.612336 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:40731: BlockingRecv error: recv got EOF from 127.15.232.195:40731 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:42 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:42.617426 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:42.612666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:56467)
0810 06:45:42.612727 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:42.612789 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:42.612847 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:42.612974 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:42.612975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:42.613080 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:42.613107 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.613108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.613761 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.613762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.614643 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:42.614646 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.615158 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.615158 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:42.615202 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.615553 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:42.615557 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:42.617334 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:45:42.617465 26319 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:42.612741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:56467 (local address 127.15.232.254:34471)
0810 06:45:42.612795 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:42.612797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:42.612834 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:45:42.612887 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:42.612888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:42.612911 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:42.612957 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:42.613205 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.613661 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.614741 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:42.615075 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:42.615137 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:42.617404 (+ 2267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:56467: BlockingRecv error: recv got EOF from 127.15.232.195:56467 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260810 06:45:42.617666 26620 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 06:45:43 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:43 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:43.348620 26624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:43.343497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:54829)
0810 06:45:43.343632 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:43.343643 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:43.343695 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:43.343909 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:43.343909 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:43.344098 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:43.344150 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:43.344152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:43.344822 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:43.344823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:43.345485 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:43.345489 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:43.345858 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:43.345859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:43.345901 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:43.346477 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:43.346482 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:43.348544 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
W20260810 06:45:43.348879 26625 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:43.343591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:54829 (local address 127.15.232.254:34471)
0810 06:45:43.343751 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:43.343753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:43.343762 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:43.343786 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:43.343787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:43.343796 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:43.343899 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:43.344226 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:43.344746 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:43.345581 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:43.345786 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:43.345846 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:43.348766 (+ 2920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:54829: BlockingRecv error: recv got EOF from 127.15.232.193:54829 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
Aug 10 06:45:43 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:43 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:43.494957 26626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:43.489688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:39781)
0810 06:45:43.489816 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:43.489828 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:43.489919 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:43.490158 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:43.490159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:43.490345 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:43.490397 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:43.490399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:43.491117 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:43.491118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:43.491670 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:43.491672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:43.492055 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:43.492056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:43.492097 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:43.492646 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:43.492656 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:43.494884 (+ 2228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
W20260810 06:45:43.495102 26625 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:43.489756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:39781 (local address 127.15.232.254:34471)
0810 06:45:43.489856 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:43.489858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:43.489917 (+ 59us) server_negotiation.cc:407] Connection header received
0810 06:45:43.490059 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:43.490060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:43.490074 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:43.490141 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:43.490463 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:43.491040 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:43.491757 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:43.491982 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:43.492041 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:43.495013 (+ 2972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:39781: BlockingRecv error: recv got EOF from 127.15.232.194:39781 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 06:45:43 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:43 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:43.623761 26627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:43.618348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:37291)
0810 06:45:43.618633 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:43.618647 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:43.618726 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:43.618941 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:43.618943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:43.619158 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:43.619210 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:43.619212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:43.619929 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:43.619930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:43.620537 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:43.620540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:43.620962 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:43.620963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:43.621038 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:43.621626 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:43.621638 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:43.623684 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":89,"threads_started":1}
W20260810 06:45:43.623951 26625 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:43.618597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:37291 (local address 127.15.232.254:34471)
0810 06:45:43.618727 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:43.618729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:43.618767 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:45:43.618819 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:43.618820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:43.618835 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:43.618927 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:43.619319 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:43.619833 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:43.620657 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:43.620862 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:43.620962 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:43.623870 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:37291: BlockingRecv error: recv got EOF from 127.15.232.195:37291 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 10 06:45:44 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:44 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:44.354413 26628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:44.349244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:52567)
0810 06:45:44.349364 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:44.349383 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:44.349463 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:44.349664 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:44.349664 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:44.349838 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:44.349887 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:44.349889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:44.350602 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:44.350603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:44.351128 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:44.351129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:44.351534 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:44.351535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:44.351597 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:44.352149 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:44.352154 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:44.354315 (+ 2161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
W20260810 06:45:44.354477 26629 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:44.349345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:52567 (local address 127.15.232.254:34471)
0810 06:45:44.349511 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:44.349514 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:44.349526 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:45:44.349563 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:44.349563 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:44.349574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:44.349657 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:44.349963 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:44.350521 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:44.351232 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:44.351451 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:44.351534 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:44.354412 (+ 2878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:52567: BlockingRecv error: recv got EOF from 127.15.232.193:52567 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
Aug 10 06:45:44 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:44 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:44.501008 26630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:44.495553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:48983)
0810 06:45:44.495768 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:44.495781 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:44.495878 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:44.496037 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:44.496038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:44.496228 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:44.496277 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:44.496279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:44.497021 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:44.497022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:44.497804 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:44.497807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:44.498165 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:44.498166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:44.498228 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:44.498743 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:44.498749 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:44.500947 (+ 2198us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":130,"threads_started":1}
W20260810 06:45:44.501223 26629 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:44.495676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:48983 (local address 127.15.232.254:34471)
0810 06:45:44.495769 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:44.495771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:44.495863 (+ 92us) server_negotiation.cc:407] Connection header received
0810 06:45:44.495941 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:44.495943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:44.495963 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:44.496029 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:44.496371 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:44.496901 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:44.497883 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:44.498082 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:44.498160 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:44.501101 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:48983: BlockingRecv error: recv got EOF from 127.15.232.194:48983 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 06:45:44 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:44 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:44.629184 26631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:44.624295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:43261)
0810 06:45:44.624496 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:44.624511 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:44.624611 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:44.624785 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:44.624786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:44.624981 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:44.625030 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:44.625032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:44.625771 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:44.625772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:44.626327 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:44.626329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:44.626735 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:44.626736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:44.626793 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:44.627315 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:44.627320 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:44.629065 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":83,"threads_started":1}
W20260810 06:45:44.629263 26629 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:44.624555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:43261 (local address 127.15.232.254:34471)
0810 06:45:44.624635 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:44.624638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:44.624661 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:45:44.624689 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:44.624690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:44.624709 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:44.624777 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:44.625120 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:44.625680 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:44.626429 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:44.626640 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:44.626724 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:44.629194 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:43261: BlockingRecv error: recv got EOF from 127.15.232.195:43261 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 06:45:45 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:45 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:45.360571 26635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:45.355050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:38813)
0810 06:45:45.355207 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:45.355225 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:45.355284 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:45.355449 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:45.355450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:45.355689 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:45.355741 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:45.355743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:45.356837 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:45.356838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:45.357524 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:45.357527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:45.357921 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:45.357923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:45.357975 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:45.358472 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:45.358477 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:45.360508 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
W20260810 06:45:45.360706 26636 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:45.355155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:38813 (local address 127.15.232.254:34471)
0810 06:45:45.355298 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:45.355300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:45.355309 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:45.355336 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:45.355337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:45.355347 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:45.355422 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:45.355843 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:45.356744 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:45.357622 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:45.357835 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:45.357895 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:45.360615 (+ 2720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:38813: BlockingRecv error: recv got EOF from 127.15.232.193:38813 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
Aug 10 06:45:45 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:45 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:45.507067 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:45.501604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:53505)
0810 06:45:45.501808 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:45.501818 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:45.501905 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:45.502115 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:45.502116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:45.502317 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:45.502366 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:45.502368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:45.503085 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:45.503087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:45.503714 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:45.503717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:45.504108 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:45.504110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:45.504180 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:45.504738 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:45.504744 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:45.507006 (+ 2262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":118,"threads_started":1}
W20260810 06:45:45.507264 26636 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:45.501689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:53505 (local address 127.15.232.254:34471)
0810 06:45:45.501761 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:45.501763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:45.501884 (+ 121us) server_negotiation.cc:407] Connection header received
0810 06:45:45.502015 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:45.502016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:45.502038 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:45.502105 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:45.502458 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:45.503000 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:45.503800 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:45.504014 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:45.504099 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:45.507189 (+ 3090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:53505: BlockingRecv error: recv got EOF from 127.15.232.194:53505 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 06:45:45 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:45 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:45.635072 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:45.629816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:34397)
0810 06:45:45.629991 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:45.630013 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:45.630090 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:45.630246 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:45.630247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:45.630419 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:45.630486 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:45.630488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:45.631172 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:45.631173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:45.631733 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:45.631735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:45.632261 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:45.632262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:45.632309 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:45.632884 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:45.632888 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:45.634989 (+ 2101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":67,"threads_started":1}
W20260810 06:45:45.635223 26636 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:45.629945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:34397 (local address 127.15.232.254:34471)
0810 06:45:45.630026 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:45.630028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:45.630086 (+ 58us) server_negotiation.cc:407] Connection header received
0810 06:45:45.630149 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:45.630150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:45.630163 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:45.630222 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:45.630554 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:45.631081 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:45.631972 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:45.632182 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:45.632251 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:45.635162 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:34397: BlockingRecv error: recv got EOF from 127.15.232.195:34397 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 06:45:46 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:46 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:46.371974 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:46.361221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:42589)
0810 06:45:46.361392 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:46.361414 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:46.361511 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:46.361927 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:46.361928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:46.362114 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:46.362166 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:46.362167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:46.362940 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:46.362941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:46.363509 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:46.363512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:46.363916 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:46.363917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:46.363971 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:46.369466 (+ 5495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:46.369473 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:46.371907 (+ 2434us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
W20260810 06:45:46.372193 26640 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:46.361618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:42589 (local address 127.15.232.254:34471)
0810 06:45:46.361752 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:46.361755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:46.361763 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:46.361791 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:46.361792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:46.361803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:46.361873 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:46.362258 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:46.362796 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:46.363614 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:46.363818 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:46.363881 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:46.372108 (+ 8227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:42589: BlockingRecv error: recv got EOF from 127.15.232.193:42589 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
Aug 10 06:45:46 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:46 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:46.513070 26641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:46.507765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:35287)
0810 06:45:46.508035 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:46.508049 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:46.508160 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:46.508333 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:46.508334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:46.508547 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:46.508599 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:46.508602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:46.509372 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:46.509374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:46.509918 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:46.509920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:46.510319 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:46.510320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:46.510383 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:46.510957 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:46.510963 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:46.512966 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":122,"threads_started":1}
W20260810 06:45:46.513182 26640 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:46.507912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:35287 (local address 127.15.232.254:34471)
0810 06:45:46.508017 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:46.508021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:46.508144 (+ 123us) server_negotiation.cc:407] Connection header received
0810 06:45:46.508232 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:46.508233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:46.508249 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:46.508325 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:46.508711 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:46.509287 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:46.510027 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:46.510238 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:46.510321 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:46.513075 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:35287: BlockingRecv error: recv got EOF from 127.15.232.194:35287 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 06:45:46 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:46 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:46.641884 26642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:46.636004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:43029)
0810 06:45:46.636200 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:46.636212 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:46.636315 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:46.636545 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:46.636546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:46.636775 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:46.636864 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:46.636867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:46.637830 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:46.637831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:46.638387 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:46.638388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:46.638978 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:46.638979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:46.639054 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:46.639564 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:46.639569 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:46.641784 (+ 2215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":112,"threads_started":1}
W20260810 06:45:46.641950 26640 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:46.636148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:43029 (local address 127.15.232.254:34471)
0810 06:45:46.636234 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:46.636236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:46.636314 (+ 78us) server_negotiation.cc:407] Connection header received
0810 06:45:46.636370 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:46.636371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:46.636385 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:46.636455 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:46.636966 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:46.637659 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:46.638478 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:46.638806 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:46.638950 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:46.641884 (+ 2934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:43029: BlockingRecv error: recv got EOF from 127.15.232.195:43029 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 06:45:47 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:47 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:47.378211 26643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:47.372613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:36575)
0810 06:45:47.372875 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:47.372895 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:47.372964 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:47.373191 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:47.373192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:47.373375 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:47.373425 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:47.373428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:47.374233 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:47.374234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:47.374832 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:47.374834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:47.375350 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:47.375352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:47.375415 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:47.376008 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:47.376014 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:47.378142 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":96,"threads_started":1}
W20260810 06:45:47.378484 26645 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:47.372758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:36575 (local address 127.15.232.254:34471)
0810 06:45:47.372956 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:47.372959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:47.372970 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:47.373021 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:47.373021 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:47.373033 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:47.373110 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:47.373526 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:47.374135 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:47.375015 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:47.375231 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:47.375301 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:47.378366 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:36575: BlockingRecv error: recv got EOF from 127.15.232.193:36575 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":74,"threads_started":1}
Aug 10 06:45:47 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:47 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:47.519310 26646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:47.513958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:49221)
0810 06:45:47.514142 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:47.514175 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:47.514251 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:47.514450 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:47.514451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:47.514639 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:47.514690 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:47.514695 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:47.515640 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:47.515642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:47.516200 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:47.516202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:47.516587 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:47.516588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:47.516630 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:47.517248 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:47.517259 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:47.519249 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":80,"threads_started":1}
W20260810 06:45:47.519484 26645 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:47.514133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:49221 (local address 127.15.232.254:34471)
0810 06:45:47.514213 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:47.514216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:47.514239 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:45:47.514350 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:47.514352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:47.514365 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:47.514444 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:47.514873 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:47.515532 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:47.516306 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:47.516510 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:47.516572 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:47.519409 (+ 2837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:49221: BlockingRecv error: recv got EOF from 127.15.232.194:49221 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 06:45:47 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:47 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:47.648298 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:47.642545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:60333)
0810 06:45:47.642862 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:47.642875 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:47.643003 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:47.643253 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:47.643254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:47.643479 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:47.643537 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:47.643539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:47.644374 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:47.644376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:47.644989 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:47.644992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:47.645515 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:47.645517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:47.645600 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:47.646181 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:47.646186 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:47.648235 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":93,"threads_started":1}
W20260810 06:45:47.648461 26645 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:47.642678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:60333 (local address 127.15.232.254:34471)
0810 06:45:47.642766 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:47.642768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:47.642962 (+ 194us) server_negotiation.cc:407] Connection header received
0810 06:45:47.643125 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:47.643126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:47.643141 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:47.643241 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:47.643638 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:47.644252 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:47.645113 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:47.645407 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:47.645513 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:47.648374 (+ 2861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:60333: BlockingRecv error: recv got EOF from 127.15.232.195:60333 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 06:45:48 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:48 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:48.384506 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:48.379024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:40107)
0810 06:45:48.379168 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:48.379183 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:48.379248 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:48.379452 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:48.379453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:48.379628 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:48.379700 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:48.379701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:48.380664 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:48.380665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:48.381379 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:48.381381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:48.381880 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:48.381881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:48.381937 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:48.382434 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:48.382439 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:48.384441 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
W20260810 06:45:48.384693 26649 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:48.379100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:40107 (local address 127.15.232.254:34471)
0810 06:45:48.379315 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:48.379318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:48.379326 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:48.379351 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:48.379352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:48.379363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:48.379429 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:48.379818 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:48.380565 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:48.381464 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:48.381680 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:48.381745 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:48.384586 (+ 2841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:40107: BlockingRecv error: recv got EOF from 127.15.232.193:40107 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
Aug 10 06:45:48 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:48 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:48.525352 26650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:48.519984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:48925)
0810 06:45:48.520159 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:48.520170 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:48.520260 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:48.520431 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:48.520432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:48.520620 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:48.520675 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:48.520677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:48.521447 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:48.521448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:48.522014 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:48.522017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:48.522412 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:48.522413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:48.522460 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:48.522954 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:48.522959 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:48.525226 (+ 2267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
W20260810 06:45:48.525511 26649 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:48.520090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:48925 (local address 127.15.232.254:34471)
0810 06:45:48.520183 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:48.520185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:48.520260 (+ 75us) server_negotiation.cc:407] Connection header received
0810 06:45:48.520318 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:48.520319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:48.520331 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:48.520393 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:48.520798 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:48.521360 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:48.522113 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:48.522332 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:48.522397 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:48.525367 (+ 2970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:48925: BlockingRecv error: recv got EOF from 127.15.232.194:48925 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 06:45:48 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:48 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:48.653898 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:48.648795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:42333)
0810 06:45:48.649028 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:48.649042 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:48.649129 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:48.649389 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:48.649390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:48.649571 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:48.649621 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:48.649623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:48.650296 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:48.650297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:48.650862 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:48.650865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:48.651239 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:48.651240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:48.651286 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:48.651747 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:48.651752 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:48.653816 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":120,"threads_started":1}
W20260810 06:45:48.653973 26649 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:48.648904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:42333 (local address 127.15.232.254:34471)
0810 06:45:48.649011 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:48.649013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:48.649126 (+ 113us) server_negotiation.cc:407] Connection header received
0810 06:45:48.649273 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:48.649274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:48.649288 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:48.649334 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:48.649699 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:48.650201 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:48.650959 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:48.651147 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:48.651207 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:48.653920 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:42333: BlockingRecv error: recv got EOF from 127.15.232.195:42333 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 10 06:45:49 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:49 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:49.394582 26653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:49.387390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:49555)
0810 06:45:49.387585 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:49.387599 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:49.387694 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:49.388077 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:49.388079 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:49.388346 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:49.388431 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:49.388435 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:49.389644 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:49.389645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:49.390816 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:49.390822 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:49.391307 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:49.391309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:49.391390 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:49.391975 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:49.391981 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:49.394515 (+ 2534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":111,"threads_started":1}
W20260810 06:45:49.394738 26654 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:49.387508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:49555 (local address 127.15.232.254:34471)
0810 06:45:49.387630 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:49.387633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:49.387685 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:45:49.387800 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:49.387802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:49.387815 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:49.387916 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:49.388541 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:49.389358 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:49.390923 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:49.391226 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:49.391302 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:49.394642 (+ 3340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:49555: BlockingRecv error: recv got EOF from 127.15.232.193:49555 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
Aug 10 06:45:49 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:49 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:49.531548 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:49.525988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:47705)
0810 06:45:49.526138 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:49.526157 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:49.526218 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:49.526392 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:49.526393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:49.526590 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:49.526644 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:49.526646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:49.527366 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:49.527368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:49.528101 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:49.528104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:49.528489 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:49.528491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:49.528548 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:49.529270 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:49.529276 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:49.531442 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
W20260810 06:45:49.531605 26654 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:49.526069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:47705 (local address 127.15.232.254:34471)
0810 06:45:49.526161 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:49.526163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:49.526207 (+ 44us) server_negotiation.cc:407] Connection header received
0810 06:45:49.526281 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:49.526282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:49.526292 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:49.526336 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:49.526728 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:49.527235 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:49.528192 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:49.528393 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:49.528482 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:49.531538 (+ 3056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:47705: BlockingRecv error: recv got EOF from 127.15.232.194:47705 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 06:45:49 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:49 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:49.660188 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:49.654589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:50903)
0810 06:45:49.654850 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:49.654866 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:49.654990 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:49.655168 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:49.655169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:49.655367 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:49.655420 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:49.655422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:49.656153 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:49.656155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:49.656867 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:49.656869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:49.657350 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:49.657351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:49.657420 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:49.657908 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:49.657913 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:49.660084 (+ 2171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":128,"threads_started":1}
W20260810 06:45:49.660244 26654 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:49.654741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:50903 (local address 127.15.232.254:34471)
0810 06:45:49.654856 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:49.654858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:49.654973 (+ 115us) server_negotiation.cc:407] Connection header received
0810 06:45:49.655055 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:49.655056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:49.655072 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:49.655161 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:49.655547 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:49.656061 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:49.656973 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:49.657262 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:49.657337 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:49.660178 (+ 2841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:50903: BlockingRecv error: recv got EOF from 127.15.232.195:50903 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 10 06:45:50 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:50 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:50.400461 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:50.395189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:33567)
0810 06:45:50.395389 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:50.395418 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:50.395479 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:50.395717 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:50.395719 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:50.395898 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:50.395953 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:50.395955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.396678 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.396679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:50.397333 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:50.397337 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.397755 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.397756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:50.397827 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:50.398370 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:50.398375 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:50.400387 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":63,"threads_started":1}
W20260810 06:45:50.400682 26659 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:50.395408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:33567 (local address 127.15.232.254:34471)
0810 06:45:50.395553 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:50.395555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:50.395564 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:50.395598 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:50.395598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:50.395608 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:50.395670 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:50.396034 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.396570 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.397435 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.397638 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.397699 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:50.400585 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:33567: BlockingRecv error: recv got EOF from 127.15.232.193:33567 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
Aug 10 06:45:50 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:50 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:50.537664 26660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:50.532193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:33177)
0810 06:45:50.532445 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:50.532457 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:50.532557 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:50.532741 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:50.532742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:50.532951 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:50.533002 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:50.533005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.533931 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.533932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:50.534473 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:50.534475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.534852 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.534853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:50.534914 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:50.535435 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:50.535450 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:50.537591 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":147,"threads_started":1}
W20260810 06:45:50.537920 26659 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:50.532314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:33177 (local address 127.15.232.254:34471)
0810 06:45:50.532417 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:50.532420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:50.532540 (+ 120us) server_negotiation.cc:407] Connection header received
0810 06:45:50.532625 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:50.532627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:50.532640 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:50.532729 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:50.533098 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.533820 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.534573 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.534775 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.534852 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:50.537837 (+ 2985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:33177: BlockingRecv error: recv got EOF from 127.15.232.194:33177 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 06:45:50 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:50 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:50.666062 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:50.660975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:51627)
0810 06:45:50.661136 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:50.661190 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:50.661264 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:50.661424 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:50.661425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:50.661608 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:50.661662 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:50.661664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.662343 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.662345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:50.662894 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:50.662896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.663234 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.663235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:50.663278 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:50.663743 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:50.663748 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:50.665956 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
W20260810 06:45:50.666134 26659 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:50.661089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:51627 (local address 127.15.232.254:34471)
0810 06:45:50.661178 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:50.661180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:50.661274 (+ 94us) server_negotiation.cc:407] Connection header received
0810 06:45:50.661333 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:50.661334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:50.661349 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:50.661402 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:50.661734 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.662238 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.662981 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.663158 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.663209 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:50.666063 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:51627: BlockingRecv error: recv got EOF from 127.15.232.195:51627 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 06:45:51 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:51 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:51.412918 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:51.401264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:37299)
0810 06:45:51.401479 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:51.401517 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:51.401598 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:51.401919 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:51.401920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:51.402174 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:51.402225 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:51.402228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.402948 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.402949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:51.403553 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:51.403555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.403989 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.403990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:51.404051 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:51.409783 (+ 5732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:51.409794 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:51.412814 (+ 3020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":104,"threads_started":1}
W20260810 06:45:51.413069 26664 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:51.401372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:37299 (local address 127.15.232.254:34471)
0810 06:45:51.401535 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:51.401537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:51.401601 (+ 64us) server_negotiation.cc:407] Connection header received
0810 06:45:51.401729 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:51.401730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:51.401749 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:51.401896 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:51.402322 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.402849 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.403668 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.403892 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.403987 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:51.412923 (+ 8936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:37299: BlockingRecv error: recv got EOF from 127.15.232.193:37299 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
Aug 10 06:45:51 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:51 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:51.543640 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:51.538246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:59851)
0810 06:45:51.538530 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:51.538541 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:51.538630 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:51.538821 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:51.538822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:51.539056 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:51.539106 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:51.539109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.539841 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.539842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:51.540486 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:51.540489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.540878 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.540879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:51.540940 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:51.541520 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:51.541525 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:51.543540 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":153,"threads_started":1}
W20260810 06:45:51.543706 26664 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:51.538367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:59851 (local address 127.15.232.254:34471)
0810 06:45:51.538484 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:51.538487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:51.538609 (+ 122us) server_negotiation.cc:407] Connection header received
0810 06:45:51.538700 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:51.538701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:51.538731 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:51.538810 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:51.539194 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.539743 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.540578 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.540780 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.540876 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:51.543642 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:59851: BlockingRecv error: recv got EOF from 127.15.232.194:59851 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 10 06:45:51 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:51 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:51.672179 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:51.666740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:53731)
0810 06:45:51.666919 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:51.666933 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:51.667007 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:51.667230 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:51.667232 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:51.667441 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:51.667489 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:51.667491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.668169 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.668169 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:51.668718 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:51.668720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.669298 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.669300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:51.669370 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:51.670066 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:51.670074 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:51.672108 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
W20260810 06:45:51.672360 26664 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:51.666896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:53731 (local address 127.15.232.254:34471)
0810 06:45:51.666948 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:51.666951 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:51.666988 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:45:51.667055 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:51.667056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:51.667069 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:51.667120 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:51.667575 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.668096 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.668908 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.669189 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.669299 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:51.672241 (+ 2942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:53731: BlockingRecv error: recv got EOF from 127.15.232.195:53731 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 10 06:45:52 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:52 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:52.419006 26667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:52.413572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:37717)
0810 06:45:52.413743 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:52.413755 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:52.413822 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:52.414134 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:52.414136 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:52.414392 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:52.414458 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:52.414461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.415244 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.415245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:52.415829 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:52.415833 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.416304 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.416304 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:52.416359 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:52.416885 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:52.416890 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:52.418929 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
W20260810 06:45:52.419346 26668 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:52.413662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:37717 (local address 127.15.232.254:34471)
0810 06:45:52.413877 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:52.413880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:52.413895 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:45:52.413996 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:52.413996 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:52.414025 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:52.414098 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:52.414546 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.415102 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.415979 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.416189 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.416259 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:52.419262 (+ 3003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:37717: BlockingRecv error: recv got EOF from 127.15.232.193:37717 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":56,"threads_started":1}
Aug 10 06:45:52 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:52 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:52.549888 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:52.544285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:44179)
0810 06:45:52.544474 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:52.544490 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:52.544573 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:52.544845 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:52.544846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:52.545020 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:52.545069 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:52.545071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.545814 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.545815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:52.546346 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:52.546348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.546745 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.546746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:52.546820 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:52.547361 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:52.547366 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:52.549775 (+ 2409us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
W20260810 06:45:52.549950 26668 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:52.544426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:44179 (local address 127.15.232.254:34471)
0810 06:45:52.544503 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:52.544505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:52.544566 (+ 61us) server_negotiation.cc:407] Connection header received
0810 06:45:52.544646 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:52.544648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:52.544661 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:52.544750 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:52.545198 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.545712 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.546437 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.546652 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.546743 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:52.549885 (+ 3142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:44179: BlockingRecv error: recv got EOF from 127.15.232.194:44179 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 06:45:52 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:52 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:52.677879 26670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:52.672813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:43353)
0810 06:45:52.673034 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:52.673046 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:52.673107 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:52.673314 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:52.673315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:52.673517 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:52.673569 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:52.673571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.674238 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.674239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:52.674775 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:52.674777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.675280 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.675281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:52.675328 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:52.675857 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:52.675862 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:52.677816 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":86,"threads_started":1}
W20260810 06:45:52.678115 26668 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:52.672939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:43353 (local address 127.15.232.254:34471)
0810 06:45:52.673026 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:52.673028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:52.673092 (+ 64us) server_negotiation.cc:407] Connection header received
0810 06:45:52.673190 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:52.673190 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:52.673227 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:52.673288 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:52.673653 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.674159 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.674930 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.675141 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.675215 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:52.678048 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:43353: BlockingRecv error: recv got EOF from 127.15.232.195:43353 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 06:45:53 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:53 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:53.424682 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.419575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:47639)
0810 06:45:53.419755 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:53.419770 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:53.419845 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:53.420115 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:53.420116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:53.420318 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:53.420369 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.420372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.421084 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.421085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.421660 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.421663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.422049 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.422050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.422117 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.422646 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:53.422651 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:53.424572 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
W20260810 06:45:53.424746 26673 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:53.419685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:47639 (local address 127.15.232.254:34471)
0810 06:45:53.419828 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.419830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.419841 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:53.419985 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.419986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.420000 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.420104 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:53.420465 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.420994 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.421756 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.421967 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.422048 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.424664 (+ 2616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:47639: BlockingRecv error: recv got EOF from 127.15.232.193:47639 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
Aug 10 06:45:53 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:53 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:53.556156 26674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.550496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:39073)
0810 06:45:53.550718 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:53.550731 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:53.550909 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:53.551102 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:53.551103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:53.551309 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:53.551356 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.551358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.552115 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.552116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.552785 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.552789 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.553325 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.553326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.553387 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.553899 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:53.553905 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:53.556083 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":94,"threads_started":1}
W20260810 06:45:53.556320 26673 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:53.550612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:39073 (local address 127.15.232.254:34471)
0810 06:45:53.550715 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.550718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.550817 (+ 99us) server_negotiation.cc:407] Connection header received
0810 06:45:53.550998 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.550999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.551021 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.551093 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:53.551448 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.551955 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.552896 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.553219 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.553294 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.556221 (+ 2927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:39073: BlockingRecv error: recv got EOF from 127.15.232.194:39073 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 06:45:53 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:53 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:53.683503 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.678408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:51775)
0810 06:45:53.678569 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:53.678579 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:53.678639 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:53.678755 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:53.678756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:53.678953 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:53.679072 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.679075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.679773 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.679774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.680416 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.680420 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.680744 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.680745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.680800 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.681329 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:53.681334 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:53.683369 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
W20260810 06:45:53.683530 26673 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:53.678460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:51775 (local address 127.15.232.254:34471)
0810 06:45:53.678540 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.678543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.678640 (+ 97us) server_negotiation.cc:407] Connection header received
0810 06:45:53.678684 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.678684 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.678697 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.678749 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:53.679153 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.679669 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.680493 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.680673 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.680727 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.683463 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:51775: BlockingRecv error: recv got EOF from 127.15.232.195:51775 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 06:45:54 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:54 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:54.430500 26676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:54.425353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:37659)
0810 06:45:54.425536 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:54.425557 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:54.425625 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:54.425792 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:54.425792 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:54.426021 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:54.426068 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:54.426071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:54.426899 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:54.426900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:54.427463 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:54.427465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:54.427859 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:54.427860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:54.427901 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:54.428423 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:54.428428 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:54.430438 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":79,"threads_started":1}
W20260810 06:45:54.430646 26677 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:54.425430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:37659 (local address 127.15.232.254:34471)
0810 06:45:54.425599 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:54.425601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:54.425612 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:45:54.425685 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:54.425686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:54.425696 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:54.425772 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:54.426171 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:54.426807 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:54.427571 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:54.427776 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:54.427848 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:54.430551 (+ 2703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:37659: BlockingRecv error: recv got EOF from 127.15.232.193:37659 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":72,"threads_started":1}
Aug 10 06:45:54 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:54 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:54.562189 26679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:54.556805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:33853)
0810 06:45:54.556982 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:54.556993 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:54.557053 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:54.557238 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:54.557239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:54.557444 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:54.557492 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:54.557494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:54.558253 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:54.558254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:54.558863 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:54.558865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:54.559286 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:54.559287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:54.559343 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:54.559961 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:54.559967 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:54.562124 (+ 2157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
W20260810 06:45:54.562399 26677 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:54.556961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:33853 (local address 127.15.232.254:34471)
0810 06:45:54.557044 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:54.557046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:54.557070 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:45:54.557098 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:54.557098 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:54.557109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:54.557209 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:54.557592 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:54.558140 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:54.558965 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:54.559169 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:54.559247 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:54.562283 (+ 3036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:33853: BlockingRecv error: recv got EOF from 127.15.232.194:33853 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 06:45:54 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:54 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:54.689658 26681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:54.684182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:39477)
0810 06:45:54.684373 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:54.684384 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:54.684453 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:54.684603 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:54.684604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:54.684825 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:54.684882 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:54.684885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:54.685606 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:54.685607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:54.686157 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:54.686160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:54.686524 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:54.686526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:54.686585 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:54.687211 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:54.687216 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:54.689589 (+ 2373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":111,"threads_started":1}
W20260810 06:45:54.689795 26677 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:54.684265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:39477 (local address 127.15.232.254:34471)
0810 06:45:54.684353 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:54.684355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:54.684453 (+ 98us) server_negotiation.cc:407] Connection header received
0810 06:45:54.684512 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:54.684513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:54.684526 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:54.684586 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:54.684982 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:54.685519 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:54.686234 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:54.686415 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:54.686482 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:54.689720 (+ 3238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:39477: BlockingRecv error: recv got EOF from 127.15.232.195:39477 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 06:45:55 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:55 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:55.436673 26682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:55.431145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:48937)
0810 06:45:55.431347 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:55.431358 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:55.431413 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:55.431665 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:55.431666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:55.431919 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:55.431965 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:55.431967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:55.432699 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:55.432699 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:55.433447 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:55.433450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:55.433843 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:55.433844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:55.433903 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:55.434540 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:55.434545 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:55.436554 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":98,"threads_started":1}
W20260810 06:45:55.436733 26683 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:55.431225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:48937 (local address 127.15.232.254:34471)
0810 06:45:55.431411 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:55.431414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:55.431427 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:45:55.431528 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:55.431529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:55.431543 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:55.431636 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:55.432062 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:55.432599 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:55.433540 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:55.433748 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:55.433824 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:55.436653 (+ 2829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:48937: BlockingRecv error: recv got EOF from 127.15.232.193:48937 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
Aug 10 06:45:55 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:55 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:55.573529 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:55.562820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:56111)
0810 06:45:55.562988 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:55.563007 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:55.563063 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:55.563271 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:55.563272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:55.563556 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:55.563617 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:55.563619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:55.564639 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:55.564640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:55.565498 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:55.565502 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:55.565887 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:55.565888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:55.565932 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:55.571411 (+ 5479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:55.571418 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:55.573451 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
W20260810 06:45:55.573719 26683 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:55.563051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:56111 (local address 127.15.232.254:34471)
0810 06:45:55.563128 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:55.563130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:55.563138 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:45:55.563169 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:55.563170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:55.563182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:55.563249 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:55.563726 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:55.564538 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:55.565603 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:55.565809 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:55.565873 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:55.573644 (+ 7771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:56111: BlockingRecv error: recv got EOF from 127.15.232.194:56111 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 06:45:55 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:55 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:55.695202 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:55.690360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:52133)
0810 06:45:55.690628 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:55.690641 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:55.690712 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:55.690889 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:55.690890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:55.691084 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:55.691136 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:55.691139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:55.691815 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:55.691815 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:55.692340 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:55.692341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:55.692683 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:55.692684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:55.692727 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:55.693306 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:55.693311 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:55.695141 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":94,"threads_started":1}
W20260810 06:45:55.695441 26683 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:55.690520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:52133 (local address 127.15.232.254:34471)
0810 06:45:55.690574 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:55.690577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:55.690718 (+ 141us) server_negotiation.cc:407] Connection header received
0810 06:45:55.690778 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:55.690778 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:55.690795 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:55.690845 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:55.691233 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:55.691738 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:55.692422 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:55.692608 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:55.692668 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:55.695375 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:52133: BlockingRecv error: recv got EOF from 127.15.232.195:52133 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 10 06:45:56 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:56 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:56.447337 26687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:56.437383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:57715)
0810 06:45:56.437546 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:56.437560 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:56.437647 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:56.437792 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:56.437793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:56.438036 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:56.438101 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:56.438104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:56.439080 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:56.439081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:56.439628 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:56.439630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:56.440059 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:56.440060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:56.440121 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:56.445065 (+ 4944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:56.445072 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:56.447248 (+ 2176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":79,"threads_started":1}
W20260810 06:45:56.447439 26688 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:56.437461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:57715 (local address 127.15.232.254:34471)
0810 06:45:56.437612 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:56.437614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:56.437623 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:56.437700 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:56.437700 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:56.437712 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:56.437773 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:56.438192 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:56.439005 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:56.439762 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:56.439981 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:56.440059 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:56.447364 (+ 7305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:57715: BlockingRecv error: recv got EOF from 127.15.232.193:57715 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
Aug 10 06:45:56 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:56 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:56.579092 26689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:56.574062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:43703)
0810 06:45:56.574228 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:56.574245 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:56.574329 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:56.574467 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:56.574468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:56.574645 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:56.574694 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:56.574695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:56.575446 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:56.575447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:56.576017 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:56.576019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:56.576416 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:56.576417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:56.576469 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:56.576970 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:56.576975 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:56.579029 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
W20260810 06:45:56.579274 26688 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:56.574156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:43703 (local address 127.15.232.254:34471)
0810 06:45:56.574224 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:56.574226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:56.574331 (+ 105us) server_negotiation.cc:407] Connection header received
0810 06:45:56.574380 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:56.574381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:56.574392 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:56.574448 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:56.574845 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:56.575352 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:56.576121 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:56.576324 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:56.576389 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:56.579187 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:43703: BlockingRecv error: recv got EOF from 127.15.232.194:43703 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 06:45:56 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:56 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.195@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:56.700654 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:56.695720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.195:52797)
0810 06:45:56.695888 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:56.695902 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:56.695968 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:56.696107 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:56.696108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:56.696306 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:56.696358 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:56.696360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:56.697017 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:56.697019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:56.697642 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:56.697646 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:56.698037 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:56.698038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:56.698091 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:56.698618 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:56.698623 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:56.700575 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":75,"threads_started":1}
W20260810 06:45:56.700879 26688 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:56.695860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:52797 (local address 127.15.232.254:34471)
0810 06:45:56.695948 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:56.695950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:56.695960 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:45:56.696025 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:56.696026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:56.696040 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:56.696083 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:56.696430 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:56.696922 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:56.697740 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:56.697942 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:56.698002 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:56.700725 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.195:52797: BlockingRecv error: recv got EOF from 127.15.232.195:52797 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 06:45:57 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:57 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.193@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:57.453361 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:57.448089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.193:38789)
0810 06:45:57.448235 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:57.448247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:57.448307 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:57.448438 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:57.448439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:57.448608 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:57.448660 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:57.448662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.449490 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.449491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:57.450106 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:57.450110 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.450510 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.450512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:57.450570 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:57.451154 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:57.451160 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:57.453279 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
W20260810 06:45:57.453521 26692 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:57.448157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:38789 (local address 127.15.232.254:34471)
0810 06:45:57.448307 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:57.448309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:57.448318 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:45:57.448344 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:57.448345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:57.448356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:57.448420 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:57.448732 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.449397 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.450192 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.450408 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.450469 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:57.453422 (+ 2953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.193:38789: BlockingRecv error: recv got EOF from 127.15.232.193:38789 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
Aug 10 06:45:57 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:45:57 dist-test-slave-vxj2 krb5kdc[25485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.194@KRBTEST.COM for oryx/127.15.232.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:45:57.585449 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:57.579742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34471 (local address 127.15.232.194:38681)
0810 06:45:57.579967 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:57.579980 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:57.580087 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:57.580276 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:57.580277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:57.580473 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:57.580523 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:57.580525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.581361 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.581362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:57.581919 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:57.581921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.582303 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.582304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:57.582365 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:57.583108 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:57.583115 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:57.585376 (+ 2261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.254:34471: Server oryx/127.15.232.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":132,"threads_started":1}
W20260810 06:45:57.585562 26692 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:57.579868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:38681 (local address 127.15.232.254:34471)
0810 06:45:57.579954 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:57.579956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:57.580064 (+ 108us) server_negotiation.cc:407] Connection header received
0810 06:45:57.580165 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:57.580166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:57.580186 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:57.580252 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:57.580613 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.581264 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.582014 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.582222 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.582305 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:57.585491 (+ 3186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.194:38681: BlockingRecv error: recv got EOF from 127.15.232.194:38681 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:45:57.604882 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 26149
I20260810 06:45:57.612988 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 26320
I20260810 06:45:57.623365 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 26476
I20260810 06:45:57.633817 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 26069
2026-08-10T06:45:57Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17664 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T06:45:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:57Z Disabled control of system clock
I20260810 06:45:57.667510 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:39033
--webserver_interface=127.15.232.254
--webserver_port=0
--builtin_ntp_servers=127.15.232.212:41783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:39033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 06:45:57.753887 26698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:57.754108 26698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:57.754130 26698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:57.755342 26698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:57.755383 26698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:57.755396 26698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:57.755409 26698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:57.755421 26698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:57.756785 26698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:41783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:39033
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:39033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:57.757004 26698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:57.757352 26698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:57.762818 26705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:57.762909 26707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:57.762976 26698 server_base.cc:1034] running on GCE node
W20260810 06:45:57.762825 26704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:57.763383 26698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:57.763713 26698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:57.764886 26698 hybrid_clock.cc:648] HybridClock initialized: now 1786344357764871 us; error 31 us; skew 500 ppm
I20260810 06:45:57.766278 26698 webserver.cc:460] Webserver started at http://127.15.232.254:43879/ using document root <none> and password file <none>
I20260810 06:45:57.766582 26698 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:57.766661 26698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:57.766824 26698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:57.767992 26698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "3abc8345ad164f6a8ffaa4e7bb6aca11"
format_stamp: "Formatted at 2026-08-10 06:45:57 on dist-test-slave-vxj2"
server_key: "f94d24c121dfea2299df9454a34bd6da"
server_key_iv: "ff9d8eba302cb89e05cf4e46e223d566"
server_key_version: "encryptionkey@0"
I20260810 06:45:57.768455 26698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "3abc8345ad164f6a8ffaa4e7bb6aca11"
format_stamp: "Formatted at 2026-08-10 06:45:57 on dist-test-slave-vxj2"
server_key: "f94d24c121dfea2299df9454a34bd6da"
server_key_iv: "ff9d8eba302cb89e05cf4e46e223d566"
server_key_version: "encryptionkey@0"
I20260810 06:45:57.770037 26698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:57.770910 26716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:57.771260 26698 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:57.771374 26698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "3abc8345ad164f6a8ffaa4e7bb6aca11"
format_stamp: "Formatted at 2026-08-10 06:45:57 on dist-test-slave-vxj2"
server_key: "f94d24c121dfea2299df9454a34bd6da"
server_key_iv: "ff9d8eba302cb89e05cf4e46e223d566"
server_key_version: "encryptionkey@0"
I20260810 06:45:57.771483 26698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:57.782377 26698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:57.782773 26698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:57.782953 26698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:57.787124 26698 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:39033
I20260810 06:45:57.787151 26772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:39033 every 8 connection(s)
I20260810 06:45:57.787550 26698 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:57.788339 26773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:57.791077 26773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11: Bootstrap starting.
I20260810 06:45:57.791683 26773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:57.791987 26773 log.cc:824] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:57.792665 26773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11: No bootstrap required, opened a new log
I20260810 06:45:57.793009 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 26698
I20260810 06:45:57.793128 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:57.793349 16291 external_mini_cluster.cc:1428] Setting key d3670eeb0bf5c008b3f5be7e8961fcf0
I20260810 06:45:57.794808 26773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3abc8345ad164f6a8ffaa4e7bb6aca11" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39033 } }
I20260810 06:45:57.794978 26773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:57.795042 26773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3abc8345ad164f6a8ffaa4e7bb6aca11, State: Initialized, Role: FOLLOWER
I20260810 06:45:57.795202 26773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3abc8345ad164f6a8ffaa4e7bb6aca11" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39033 } }
I20260810 06:45:57.795290 26773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:57.795315 26773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:57.795379 26773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:57.795943 26773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3abc8345ad164f6a8ffaa4e7bb6aca11" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39033 } }
I20260810 06:45:57.796031 26773 leader_election.cc:302] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3abc8345ad164f6a8ffaa4e7bb6aca11; no voters:
I20260810 06:45:57.796269 26773 leader_election.cc:288] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:57.796295 26778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:57.796423 26778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [term 1 LEADER]: Becoming Leader. State: Replica: 3abc8345ad164f6a8ffaa4e7bb6aca11, State: Running, Role: LEADER
I20260810 06:45:57.796555 26773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:57.796576 26778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3abc8345ad164f6a8ffaa4e7bb6aca11" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39033 } }
I20260810 06:45:57.797751 26779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3abc8345ad164f6a8ffaa4e7bb6aca11. Latest consensus state: current_term: 1 leader_uuid: "3abc8345ad164f6a8ffaa4e7bb6aca11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3abc8345ad164f6a8ffaa4e7bb6aca11" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39033 } } }
I20260810 06:45:57.797854 26779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:57.797850 26780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3abc8345ad164f6a8ffaa4e7bb6aca11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3abc8345ad164f6a8ffaa4e7bb6aca11" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39033 } } }
I20260810 06:45:57.798362 26780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11 [sys.catalog]: This master's current role is: LEADER
W20260810 06:45:57.798437 26795 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:45:57.798481 26795 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:45:57.799008 26796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:57.799234 26796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:57.800280 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:57.793945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50432 (local address 127.15.232.254:39033)
0810 06:45:57.794162 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:57.794170 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:57.794198 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:45:57.794714 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:57.794722 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:57.794781 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:57.794950 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:57.795178 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.796700 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.797871 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.798530 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.798630 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:57.799509 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:57.799513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:57.799513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:57.799739 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:57.799783 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:57.799798 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:57.799863 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:57.799987 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 06:45:57.800075 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260810 06:45:57.800861 26796 catalog_manager.cc:1347] Generated new cluster ID: 485306a8bce74c35b05a5f6b681eb8f0
I20260810 06:45:57.800931 26796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:57.811070 26796 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:57.811666 26796 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:57.818337 26796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3abc8345ad164f6a8ffaa4e7bb6aca11: Generated new TSK 0
I20260810 06:45:57.818590 26796 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:45:57.820958 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:39033
--builtin_ntp_servers=127.15.232.212:41783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 06:45:57.919315 26799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:57.919626 26799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:57.919668 26799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:57.920919 26799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:57.921000 26799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:57.921046 26799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:57.922559 26799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:41783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:39033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:57.922863 26799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:57.923197 26799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:57.928637 26806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:57.928649 26807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:57.928644 26809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:57.929221 26799 server_base.cc:1034] running on GCE node
I20260810 06:45:57.929445 26799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:57.929733 26799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:57.930907 26799 hybrid_clock.cc:648] HybridClock initialized: now 1786344357930883 us; error 37 us; skew 500 ppm
I20260810 06:45:57.932410 26799 webserver.cc:460] Webserver started at http://127.15.232.193:42601/ using document root <none> and password file <none>
I20260810 06:45:57.932695 26799 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:57.932753 26799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:57.932924 26799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:57.934160 26799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "041f6377740e4d55aa8fdf591ff29fc4"
format_stamp: "Formatted at 2026-08-10 06:45:57 on dist-test-slave-vxj2"
server_key: "484aacfe3f262aa921e99d3b76ad5963"
server_key_iv: "ea3e6364f8510e2cec3047e6b32973d7"
server_key_version: "encryptionkey@0"
I20260810 06:45:57.934583 26799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "041f6377740e4d55aa8fdf591ff29fc4"
format_stamp: "Formatted at 2026-08-10 06:45:57 on dist-test-slave-vxj2"
server_key: "484aacfe3f262aa921e99d3b76ad5963"
server_key_iv: "ea3e6364f8510e2cec3047e6b32973d7"
server_key_version: "encryptionkey@0"
I20260810 06:45:57.936053 26799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:57.936956 26816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:57.937232 26799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:57.937340 26799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "041f6377740e4d55aa8fdf591ff29fc4"
format_stamp: "Formatted at 2026-08-10 06:45:57 on dist-test-slave-vxj2"
server_key: "484aacfe3f262aa921e99d3b76ad5963"
server_key_iv: "ea3e6364f8510e2cec3047e6b32973d7"
server_key_version: "encryptionkey@0"
I20260810 06:45:57.937430 26799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:57.952948 26799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:57.953413 26799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:57.953599 26799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:57.953887 26799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:57.954294 26799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:57.954352 26799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:57.954398 26799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:57.954423 26799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:57.960685 26799 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:35533
I20260810 06:45:57.960729 26950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:35533 every 8 connection(s)
I20260810 06:45:57.961086 26799 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:57.967190 26954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:57.961950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:39033 (local address 127.15.232.193:56479)
0810 06:45:57.962208 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:57.962266 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:57.962781 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:57.963023 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:57.963026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:57.963035 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:57.963555 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:57.963560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.964340 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.964341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:57.965384 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:57.965387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.965741 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.965743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:57.965895 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:57.966480 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:57.966481 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:57.966531 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:57.966558 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:57.966821 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:57.966828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:57.966831 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:57.966909 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:45:57.967003 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":97,"threads_started":1}
I20260810 06:45:57.967232 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:57.961951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:56479 (local address 127.15.232.254:39033)
0810 06:45:57.962028 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:57.962031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:57.962368 (+ 337us) server_negotiation.cc:407] Connection header received
0810 06:45:57.962877 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:57.962879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:57.962893 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:57.962956 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:57.963668 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.964195 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.965469 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:57.965657 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:57.965714 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:57.966648 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:57.966649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:57.966649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:57.966654 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:57.966670 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:57.966681 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:57.966726 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:57.967149 (+ 423us) server_negotiation.cc:299] Negotiation successful
0810 06:45:57.967173 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:45:57.967636 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 26799
I20260810 06:45:57.967721 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:57.967929 26951 heartbeater.cc:347] Connected to a master server at 127.15.232.254:39033
I20260810 06:45:57.967998 16291 external_mini_cluster.cc:1428] Setting key 626086d4150c00830bc3b7115c877349
I20260810 06:45:57.968043 26951 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:57.968305 26951 heartbeater.cc:511] Master 127.15.232.254:39033 requested a full tablet report, sending...
I20260810 06:45:57.968952 26734 ts_manager.cc:194] Registered new tserver with Master: 041f6377740e4d55aa8fdf591ff29fc4 (127.15.232.193:35533)
I20260810 06:45:57.969305 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:39033
--builtin_ntp_servers=127.15.232.212:41783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 06:45:57.969859 26734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.193:56479
W20260810 06:45:58.054155 26955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:58.054440 26955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:58.054479 26955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:58.055681 26955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:58.055758 26955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:58.055801 26955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:58.057233 26955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:41783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:39033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:58.057523 26955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:58.057838 26955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:58.063433 26961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:58.063400 26965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:58.063357 26962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:58.063683 26955 server_base.cc:1034] running on GCE node
I20260810 06:45:58.063937 26955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:58.064225 26955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:58.065410 26955 hybrid_clock.cc:648] HybridClock initialized: now 1786344358065402 us; error 58 us; skew 500 ppm
I20260810 06:45:58.066874 26955 webserver.cc:460] Webserver started at http://127.15.232.194:44385/ using document root <none> and password file <none>
I20260810 06:45:58.067137 26955 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:58.067214 26955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:58.067395 26955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:58.068560 26955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "bdf7a38d4a55492a8ddebf9b5d194ff2"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "5eab526289b7c29f331f240e595b742c"
server_key_iv: "c9477829a7f6ab91fad325dde34ada89"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.069047 26955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "bdf7a38d4a55492a8ddebf9b5d194ff2"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "5eab526289b7c29f331f240e595b742c"
server_key_iv: "c9477829a7f6ab91fad325dde34ada89"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.070523 26955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:58.071386 26973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.071575 26955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:58.071689 26955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "bdf7a38d4a55492a8ddebf9b5d194ff2"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "5eab526289b7c29f331f240e595b742c"
server_key_iv: "c9477829a7f6ab91fad325dde34ada89"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.071770 26955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:58.094620 26955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:58.095046 26955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:58.095227 26955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:58.095511 26955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:58.095938 26955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:58.096005 26955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.096050 26955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:58.096076 26955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.102610 26955 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:46291
I20260810 06:45:58.102656 27101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:46291 every 8 connection(s)
I20260810 06:45:58.103026 26955 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:58.105202 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 26955
I20260810 06:45:58.105314 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:58.105561 16291 external_mini_cluster.cc:1428] Setting key 74817848a39de8b519350e2473715e06
I20260810 06:45:58.106763 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:39033
--builtin_ntp_servers=127.15.232.212:41783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 06:45:58.109752 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:58.103806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:33923 (local address 127.15.232.254:39033)
0810 06:45:58.103858 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:58.103860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:58.104229 (+ 369us) server_negotiation.cc:407] Connection header received
0810 06:45:58.104825 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:58.104827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:58.104842 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:58.104900 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:58.105956 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:58.106568 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:58.107801 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:58.108130 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:58.108208 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:58.109327 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:58.109328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:58.109329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:58.109334 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:58.109362 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:58.109367 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:58.109437 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:58.109559 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 06:45:58.109627 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:45:58.110004 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:58.103802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:39033 (local address 127.15.232.194:33923)
0810 06:45:58.104085 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:58.104123 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:58.104694 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:58.104998 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:58.105003 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:58.105014 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:58.105832 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:58.105846 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:58.106663 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:58.106664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:58.107691 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:58.107694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:58.108259 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:58.108260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:58.108450 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:58.109124 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:58.109125 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:58.109190 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:58.109223 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:58.109446 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:58.109451 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:58.109454 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:58.109571 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:45:58.109675 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":86,"threads_started":1}
I20260810 06:45:58.110591 27103 heartbeater.cc:347] Connected to a master server at 127.15.232.254:39033
I20260810 06:45:58.110694 27103 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:58.110914 27103 heartbeater.cc:511] Master 127.15.232.254:39033 requested a full tablet report, sending...
I20260810 06:45:58.111414 26734 ts_manager.cc:194] Registered new tserver with Master: bdf7a38d4a55492a8ddebf9b5d194ff2 (127.15.232.194:46291)
I20260810 06:45:58.111806 26734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.194:33923
W20260810 06:45:58.192773 27107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:58.192998 27107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:58.193022 27107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:58.194290 27107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:58.194339 27107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:58.194367 27107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:58.195787 27107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:41783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:39033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:58.196089 27107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:58.196475 27107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:58.202049 27112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:58.202230 27107 server_base.cc:1034] running on GCE node
W20260810 06:45:58.202072 27113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:58.202041 27115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:58.202540 27107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:58.202812 27107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:58.204005 27107 hybrid_clock.cc:648] HybridClock initialized: now 1786344358203996 us; error 38 us; skew 500 ppm
I20260810 06:45:58.205452 27107 webserver.cc:460] Webserver started at http://127.15.232.195:37455/ using document root <none> and password file <none>
I20260810 06:45:58.205715 27107 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:58.205787 27107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:58.205976 27107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:58.207137 27107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "75afd897ff274476829b6fc57e758899"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "462e0dbe90866663f4efe42e65cde4e1"
server_key_iv: "72d5220f0fe80351efc4e5f26409d836"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.207525 27107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "75afd897ff274476829b6fc57e758899"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "462e0dbe90866663f4efe42e65cde4e1"
server_key_iv: "72d5220f0fe80351efc4e5f26409d836"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.209025 27107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:45:58.209837 27126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.210057 27107 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:45:58.210166 27107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "75afd897ff274476829b6fc57e758899"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "462e0dbe90866663f4efe42e65cde4e1"
server_key_iv: "72d5220f0fe80351efc4e5f26409d836"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.210251 27107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:58.224267 27107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:58.224644 27107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:58.224813 27107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:58.225080 27107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:58.225522 27107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:58.225581 27107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.225626 27107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:58.225651 27107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.231781 27107 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:44787
I20260810 06:45:58.231899 27255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:44787 every 8 connection(s)
I20260810 06:45:58.232249 27107 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:58.233317 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 27107
I20260810 06:45:58.233433 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:58.233659 16291 external_mini_cluster.cc:1428] Setting key 6c042794baac4c49dec5ce044fe7cecb
I20260810 06:45:58.238507 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:58.233201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:59231 (local address 127.15.232.254:39033)
0810 06:45:58.233282 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:58.233284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:58.233583 (+ 299us) server_negotiation.cc:407] Connection header received
0810 06:45:58.234253 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:58.234253 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:58.234265 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:58.234317 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:58.235243 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:58.235769 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:58.236879 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:58.237110 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:58.237256 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:58.238224 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:58.238225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:58.238225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:58.238230 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:58.238241 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:58.238244 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:58.238287 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:58.238394 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 06:45:58.238443 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:45:58.238721 27259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:58.233065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:39033 (local address 127.15.232.195:59231)
0810 06:45:58.233422 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:58.233473 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:58.234099 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:58.234421 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:58.234426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:58.234442 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:58.235135 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:58.235142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:58.235890 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:58.235891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:58.236769 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:58.236772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:58.237277 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:58.237278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:58.237446 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:58.238081 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:58.238081 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:58.238115 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:58.238143 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:58.238300 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:58.238305 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:58.238308 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:45:58.238390 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:45:58.238489 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":147,"threads_started":1}
I20260810 06:45:58.239257 27257 heartbeater.cc:347] Connected to a master server at 127.15.232.254:39033
I20260810 06:45:58.239353 27257 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:58.239562 27257 heartbeater.cc:511] Master 127.15.232.254:39033 requested a full tablet report, sending...
I20260810 06:45:58.240010 26734 ts_manager.cc:194] Registered new tserver with Master: 75afd897ff274476829b6fc57e758899 (127.15.232.195:44787)
I20260810 06:45:58.240397 26734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.195:59231
I20260810 06:45:58.245747 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:45:58.247730 26777 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:58.247278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.15.232.254:39033)
0810 06:45:58.247349 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:58.247351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:58.247449 (+ 98us) server_negotiation.cc:407] Connection header received
0810 06:45:58.247486 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:58.247487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:58.247496 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:58.247543 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:58.247670 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50446: BlockingRecv error: recv got EOF from 127.0.0.1:50446 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:45:58.247800 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 26799
I20260810 06:45:58.256824 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 26955
I20260810 06:45:58.263119 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 27107
I20260810 06:45:58.272019 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 26698
2026-08-10T06:45:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (635 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T06:45:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:58Z Disabled control of system clock
I20260810 06:45:58.302779 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:41733
--webserver_interface=127.15.232.254
--webserver_port=0
--builtin_ntp_servers=127.15.232.212:34377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:41733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:45:58.388079 27269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:58.388314 27269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:58.388339 27269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:58.389539 27269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:58.389581 27269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:58.389595 27269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:58.389607 27269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:58.389619 27269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:58.390970 27269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:34377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:41733
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:41733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:58.391201 27269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:58.391472 27269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:58.396822 27277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:58.396872 27279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:58.396835 27269 server_base.cc:1034] running on GCE node
W20260810 06:45:58.396836 27275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:58.397338 27269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:58.397643 27269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:58.398816 27269 hybrid_clock.cc:648] HybridClock initialized: now 1786344358398771 us; error 71 us; skew 500 ppm
I20260810 06:45:58.400215 27269 webserver.cc:460] Webserver started at http://127.15.232.254:44823/ using document root <none> and password file <none>
I20260810 06:45:58.400452 27269 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:58.400499 27269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:58.400600 27269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:58.401813 27269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "97a70880abc74acd90369dd82896be8f"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "64958f2a65e70e64b91ce3581b5ac4d5"
server_key_iv: "283f065ef19338d67d16f6b2997a5eac"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.402150 27269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "97a70880abc74acd90369dd82896be8f"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "64958f2a65e70e64b91ce3581b5ac4d5"
server_key_iv: "283f065ef19338d67d16f6b2997a5eac"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.403566 27269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:45:58.404521 27286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.404840 27269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:58.404932 27269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "97a70880abc74acd90369dd82896be8f"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "64958f2a65e70e64b91ce3581b5ac4d5"
server_key_iv: "283f065ef19338d67d16f6b2997a5eac"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.405001 27269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:58.408841 27269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:58.409109 27269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:58.409294 27269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:58.413745 27269 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:41733
I20260810 06:45:58.413812 27344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:41733 every 8 connection(s)
I20260810 06:45:58.414175 27269 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:58.415221 27345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:58.417725 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 27269
I20260810 06:45:58.417819 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:45:58.417820 27345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f: Bootstrap starting.
I20260810 06:45:58.418038 16291 external_mini_cluster.cc:1428] Setting key 4ebfa5004fcd244e9336c9723170eeff
I20260810 06:45:58.418577 27345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:58.418938 27345 log.cc:824] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:58.419755 27345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f: No bootstrap required, opened a new log
I20260810 06:45:58.421046 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:58.418630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50110 (local address 127.15.232.254:41733)
0810 06:45:58.418850 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:58.418861 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:58.418898 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:45:58.419439 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:58.419449 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:58.419506 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:58.419670 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:58.420453 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:58.420457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:58.420457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:58.420462 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:58.420512 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:58.420526 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:58.420630 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:58.420670 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 06:45:58.420837 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260810 06:45:58.421739 27345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a70880abc74acd90369dd82896be8f" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41733 } }
I20260810 06:45:58.421953 27345 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:58.421998 27345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a70880abc74acd90369dd82896be8f, State: Initialized, Role: FOLLOWER
I20260810 06:45:58.422123 27345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a70880abc74acd90369dd82896be8f" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41733 } }
I20260810 06:45:58.422185 27345 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:58.422207 27345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:58.422235 27345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:58.423074 27345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a70880abc74acd90369dd82896be8f" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41733 } }
I20260810 06:45:58.423170 27345 leader_election.cc:302] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97a70880abc74acd90369dd82896be8f; no voters:
I20260810 06:45:58.423308 27345 leader_election.cc:288] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:58.423467 27350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:58.423616 27345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:58.423640 27350 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [term 1 LEADER]: Becoming Leader. State: Replica: 97a70880abc74acd90369dd82896be8f, State: Running, Role: LEADER
I20260810 06:45:58.423791 27350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a70880abc74acd90369dd82896be8f" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41733 } }
I20260810 06:45:58.424223 27351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97a70880abc74acd90369dd82896be8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a70880abc74acd90369dd82896be8f" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41733 } } }
I20260810 06:45:58.424242 27352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97a70880abc74acd90369dd82896be8f. Latest consensus state: current_term: 1 leader_uuid: "97a70880abc74acd90369dd82896be8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a70880abc74acd90369dd82896be8f" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41733 } } }
I20260810 06:45:58.424355 27351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:58.424368 27352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:58.424633 27360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:58.425242 27360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:58.426961 27360 catalog_manager.cc:1347] Generated new cluster ID: e2dc9e1cef244da2a9ef6fee0ff19cc8
I20260810 06:45:58.427042 27360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:58.436453 27360 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:58.436667 27360 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:58.455962 27360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97a70880abc74acd90369dd82896be8f: Generated new TSK 0
I20260810 06:45:58.456218 27360 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:45:58.466006 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41733
--builtin_ntp_servers=127.15.232.212:34377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:45:58.553776 27372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:58.554010 27372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:58.554034 27372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:58.555269 27372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:58.555320 27372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:58.555353 27372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:45:58.556764 27372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:34377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:58.557050 27372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:58.557413 27372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:58.563122 27380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:58.563145 27379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:58.563431 27372 server_base.cc:1034] running on GCE node
W20260810 06:45:58.563145 27382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:58.563812 27372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:58.564144 27372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:58.565321 27372 hybrid_clock.cc:648] HybridClock initialized: now 1786344358565297 us; error 50 us; skew 500 ppm
I20260810 06:45:58.566860 27372 webserver.cc:460] Webserver started at http://127.15.232.193:38053/ using document root <none> and password file <none>
I20260810 06:45:58.567188 27372 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:58.567245 27372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:58.567438 27372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:58.568641 27372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "ff612896cc544746b1757dce40595c6d"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "041953125fc3d243cc0bdb4c8f1527bd"
server_key_iv: "3258960149882023ec490044d24072d7"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.569097 27372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "ff612896cc544746b1757dce40595c6d"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "041953125fc3d243cc0bdb4c8f1527bd"
server_key_iv: "3258960149882023ec490044d24072d7"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.570669 27372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:58.571663 27388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.572037 27372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:58.572167 27372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "ff612896cc544746b1757dce40595c6d"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "041953125fc3d243cc0bdb4c8f1527bd"
server_key_iv: "3258960149882023ec490044d24072d7"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.572261 27372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:58.576002 27372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:58.576431 27372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:58.576617 27372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:58.576858 27372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:58.577309 27372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:58.577351 27372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.577373 27372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:58.577427 27372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.583420 27372 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:34523
I20260810 06:45:58.583472 27521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:34523 every 8 connection(s)
I20260810 06:45:58.583811 27372 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:58.587194 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:58.584549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:58451 (local address 127.15.232.254:41733)
0810 06:45:58.584642 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:58.584645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:58.584919 (+ 274us) server_negotiation.cc:407] Connection header received
0810 06:45:58.585514 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:58.585515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:58.585521 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:58.585576 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:58.586938 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:58.586940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:58.586940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:58.586944 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:58.586958 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:58.586962 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:58.587010 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:58.587125 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 06:45:58.587140 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:45:58.587356 27525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:58.584511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41733 (local address 127.15.232.193:58451)
0810 06:45:58.584777 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:58.584814 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:58.585400 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:58.585633 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:58.585636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:58.585644 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:58.586768 (+ 1124us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:58.586769 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:58.586799 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:58.586833 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:58.587010 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:58.587013 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:58.587015 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:45:58.587080 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 06:45:58.587131 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":119,"threads_started":1}
I20260810 06:45:58.588017 27522 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41733
I20260810 06:45:58.588132 27522 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:58.588306 27522 heartbeater.cc:511] Master 127.15.232.254:41733 requested a full tablet report, sending...
I20260810 06:45:58.588930 27305 ts_manager.cc:194] Registered new tserver with Master: ff612896cc544746b1757dce40595c6d (127.15.232.193:34523)
I20260810 06:45:58.591881 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 27372
I20260810 06:45:58.591960 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:58.592127 16291 external_mini_cluster.cc:1428] Setting key 2e33793875e9f869e621f166a53f0d97
I20260810 06:45:58.593128 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41733
--builtin_ntp_servers=127.15.232.212:34377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:45:58.679626 27527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:58.679888 27527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:58.679917 27527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:58.681412 27527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:58.681469 27527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:58.681504 27527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:45:58.683120 27527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:34377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:58.683446 27527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:58.683821 27527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:58.689662 27532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:58.689654 27533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:58.689657 27536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:58.690078 27527 server_base.cc:1034] running on GCE node
I20260810 06:45:58.690297 27527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:58.690562 27527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:58.691746 27527 hybrid_clock.cc:648] HybridClock initialized: now 1786344358691741 us; error 37 us; skew 500 ppm
I20260810 06:45:58.693255 27527 webserver.cc:460] Webserver started at http://127.15.232.194:37313/ using document root <none> and password file <none>
I20260810 06:45:58.693534 27527 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:58.693585 27527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:58.693769 27527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:58.694950 27527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "76b2808aa344417bb7644cce161e5387"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "0533bb81f2eb00918b712b3df29ddb79"
server_key_iv: "eb4da755a36a2336c67c11145ace2e1a"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.695329 27527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "76b2808aa344417bb7644cce161e5387"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "0533bb81f2eb00918b712b3df29ddb79"
server_key_iv: "eb4da755a36a2336c67c11145ace2e1a"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.696794 27527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:45:58.697715 27543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.697903 27527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:45:58.698038 27527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "76b2808aa344417bb7644cce161e5387"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "0533bb81f2eb00918b712b3df29ddb79"
server_key_iv: "eb4da755a36a2336c67c11145ace2e1a"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.698150 27527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:58.701524 27527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:58.701764 27527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:58.701861 27527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:58.702025 27527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:58.702268 27527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:58.702297 27527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.702317 27527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:58.702329 27527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.708256 27527 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:36589
I20260810 06:45:58.708331 27675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:36589 every 8 connection(s)
I20260810 06:45:58.708704 27527 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:58.712158 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:58.709522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:34479 (local address 127.15.232.254:41733)
0810 06:45:58.709595 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:58.709597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:58.709895 (+ 298us) server_negotiation.cc:407] Connection header received
0810 06:45:58.710486 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:58.710487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:58.710495 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:58.710562 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:58.711843 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:58.711845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:58.711846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:58.711851 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:58.711868 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:58.711872 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:58.711940 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:58.712055 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 06:45:58.712073 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:45:58.712399 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:58.709524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41733 (local address 127.15.232.194:34479)
0810 06:45:58.709758 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:58.709795 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:58.710360 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:58.710652 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:58.710656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:58.710665 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:58.711665 (+ 1000us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:58.711666 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:58.711699 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:58.711732 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:58.711942 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:58.711948 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:58.711952 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:45:58.712068 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 06:45:58.712140 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260810 06:45:58.712916 27676 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41733
I20260810 06:45:58.713020 27676 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:58.713260 27676 heartbeater.cc:511] Master 127.15.232.254:41733 requested a full tablet report, sending...
I20260810 06:45:58.713639 27305 ts_manager.cc:194] Registered new tserver with Master: 76b2808aa344417bb7644cce161e5387 (127.15.232.194:36589)
I20260810 06:45:58.718587 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 27527
I20260810 06:45:58.718683 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:58.718959 16291 external_mini_cluster.cc:1428] Setting key 2f1991abd8c12abba15b0117d8b7f153
I20260810 06:45:58.720093 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41733
--builtin_ntp_servers=127.15.232.212:34377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:45:58.805200 27679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:58.805413 27679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:58.805441 27679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:58.806661 27679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:58.806706 27679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:58.806733 27679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:45:58.808235 27679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:34377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:45:58.808496 27679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:58.808820 27679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:58.814054 27685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:58.814211 27688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:58.814225 27686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:58.814488 27679 server_base.cc:1034] running on GCE node
I20260810 06:45:58.814749 27679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:58.815062 27679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:58.816238 27679 hybrid_clock.cc:648] HybridClock initialized: now 1786344358816233 us; error 46 us; skew 500 ppm
I20260810 06:45:58.817684 27679 webserver.cc:460] Webserver started at http://127.15.232.195:37155/ using document root <none> and password file <none>
I20260810 06:45:58.817965 27679 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:58.818039 27679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:58.818203 27679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:58.819310 27679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "bdfb43ca0c154b84805e5a6e259a28b5"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "a2f6d4a44e505c7f695d03857d56d20a"
server_key_iv: "61ceea50b69bd866b2753ea92509741a"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.819738 27679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "bdfb43ca0c154b84805e5a6e259a28b5"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "a2f6d4a44e505c7f695d03857d56d20a"
server_key_iv: "61ceea50b69bd866b2753ea92509741a"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.821147 27679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:45:58.822109 27694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.822324 27679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:45:58.822454 27679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "bdfb43ca0c154b84805e5a6e259a28b5"
format_stamp: "Formatted at 2026-08-10 06:45:58 on dist-test-slave-vxj2"
server_key: "a2f6d4a44e505c7f695d03857d56d20a"
server_key_iv: "61ceea50b69bd866b2753ea92509741a"
server_key_version: "encryptionkey@0"
I20260810 06:45:58.822546 27679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:58.826354 27679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:58.826651 27679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:58.826794 27679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:58.827003 27679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:58.827332 27679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:58.827384 27679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.827425 27679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:58.827461 27679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:58.833746 27679 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:33553
I20260810 06:45:58.833789 27827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:33553 every 8 connection(s)
I20260810 06:45:58.834203 27679 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:58.835328 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 27679
I20260810 06:45:58.835407 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:58.835623 16291 external_mini_cluster.cc:1428] Setting key 88dcfe8e647a7655437729af577cf820
I20260810 06:45:58.837852 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:58.835085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:37585 (local address 127.15.232.254:41733)
0810 06:45:58.835179 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:58.835181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:58.835485 (+ 304us) server_negotiation.cc:407] Connection header received
0810 06:45:58.836206 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:58.836206 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:58.836213 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:58.836260 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:58.837622 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:58.837623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:58.837623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:58.837627 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:58.837642 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:58.837646 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:58.837693 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:58.837787 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 06:45:58.837799 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:45:58.838009 27830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:58.835070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41733 (local address 127.15.232.195:37585)
0810 06:45:58.835281 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:58.835324 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:58.836113 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:58.836334 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:58.836338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:58.836366 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:58.837480 (+ 1114us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:58.837480 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:58.837511 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:58.837540 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:58.837692 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:58.837694 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:58.837696 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:45:58.837747 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 06:45:58.837797 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260810 06:45:58.838426 27828 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41733
I20260810 06:45:58.838531 27828 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:58.838685 27828 heartbeater.cc:511] Master 127.15.232.254:41733 requested a full tablet report, sending...
I20260810 06:45:58.839123 27305 ts_manager.cc:194] Registered new tserver with Master: bdfb43ca0c154b84805e5a6e259a28b5 (127.15.232.195:33553)
I20260810 06:45:58.847971 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:45:58.850296 27348 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:58.849652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50112 (local address 127.15.232.254:41733)
0810 06:45:58.849723 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:58.849725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:58.849988 (+ 263us) server_negotiation.cc:407] Connection header received
0810 06:45:58.850030 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:58.850031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:58.850038 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:58.850097 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:58.850239 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50112: BlockingRecv error: recv got EOF from 127.0.0.1:50112 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:45:58.850399 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 27372
I20260810 06:45:58.860383 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 27527
I20260810 06:45:58.870296 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 27679
I20260810 06:45:58.879585 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 27269
2026-08-10T06:45:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (606 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:58 dist-test-slave-vxj2 krb5kdc[27840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:58 dist-test-slave-vxj2 krb5kdc[27840](info): set up 2 sockets
Aug 10 06:45:58 dist-test-slave-vxj2 krb5kdc[27840](info): commencing operation
krb5kdc: starting...
W20260810 06:46:00.009472 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:46:00 dist-test-slave-vxj2 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:46:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:46:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:46:00.149303 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:36991
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:36991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:00.234022 27855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:00.234325 27855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:00.234370 27855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:00.235662 27855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:46:00.235749 27855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:00.235775 27855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:00.235809 27855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:46:00.235859 27855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:46:00.237623 27855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:36991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:36991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:46:00.237963 27855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:00.238219 27855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:00.243549 27863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:00.243640 27861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:00.243549 27860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:00.243850 27855 server_base.cc:1034] running on GCE node
I20260810 06:46:00.244172 27855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:00.244515 27855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:00.245689 27855 hybrid_clock.cc:648] HybridClock initialized: now 1786344360245664 us; error 49 us; skew 500 ppm
Aug 10 06:46:00 dist-test-slave-vxj2 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344360, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:00.248541 27855 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:46:00.249073 27855 webserver.cc:460] Webserver started at http://127.15.232.254:39193/ using document root <none> and password file <none>
I20260810 06:46:00.249342 27855 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:00.249415 27855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:00.249583 27855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:46:00.250802 27855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "a2dfdc6a152042c3be53b9f0efcad633"
format_stamp: "Formatted at 2026-08-10 06:46:00 on dist-test-slave-vxj2"
server_key: "e9095152089c13b901556dd4eb00f111"
server_key_iv: "b163f68cd8497df66c835e2f7d6a2bdf"
server_key_version: "encryptionkey@0"
I20260810 06:46:00.251197 27855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "a2dfdc6a152042c3be53b9f0efcad633"
format_stamp: "Formatted at 2026-08-10 06:46:00 on dist-test-slave-vxj2"
server_key: "e9095152089c13b901556dd4eb00f111"
server_key_iv: "b163f68cd8497df66c835e2f7d6a2bdf"
server_key_version: "encryptionkey@0"
I20260810 06:46:00.252796 27855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:46:00.253927 27872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:00.254308 27855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:46:00.254449 27855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "a2dfdc6a152042c3be53b9f0efcad633"
format_stamp: "Formatted at 2026-08-10 06:46:00 on dist-test-slave-vxj2"
server_key: "e9095152089c13b901556dd4eb00f111"
server_key_iv: "b163f68cd8497df66c835e2f7d6a2bdf"
server_key_version: "encryptionkey@0"
I20260810 06:46:00.254559 27855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:00.290421 27855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:00.293015 27855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:00.293237 27855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:00.297281 27855 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:36991
I20260810 06:46:00.297343 27931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:36991 every 8 connection(s)
I20260810 06:46:00.297741 27855 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:46:00.298846 27933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:00.301402 27933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633: Bootstrap starting.
I20260810 06:46:00.301981 27933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:00.302225 27933 log.cc:824] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:00.303049 27933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633: No bootstrap required, opened a new log
I20260810 06:46:00.304595 27933 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2dfdc6a152042c3be53b9f0efcad633" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36991 } }
I20260810 06:46:00.304749 27933 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:00.304775 27933 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2dfdc6a152042c3be53b9f0efcad633, State: Initialized, Role: FOLLOWER
I20260810 06:46:00.304884 27933 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2dfdc6a152042c3be53b9f0efcad633" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36991 } }
I20260810 06:46:00.304978 27933 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:46:00.305008 27933 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:46:00.305055 27933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:00.305805 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 27855
I20260810 06:46:00.305919 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:46:00.306128 16291 external_mini_cluster.cc:1428] Setting key c3237b7822b639932b7f47fec12adb3b
I20260810 06:46:00.306135 27933 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2dfdc6a152042c3be53b9f0efcad633" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36991 } }
I20260810 06:46:00.306244 27933 leader_election.cc:302] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2dfdc6a152042c3be53b9f0efcad633; no voters:
I20260810 06:46:00.306483 27936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:46:00.306445 27933 leader_election.cc:288] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:46:00.306656 27936 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [term 1 LEADER]: Becoming Leader. State: Replica: a2dfdc6a152042c3be53b9f0efcad633, State: Running, Role: LEADER
I20260810 06:46:00.306788 27936 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2dfdc6a152042c3be53b9f0efcad633" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36991 } }
I20260810 06:46:00.306965 27933 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:46:00.307132 27936 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2dfdc6a152042c3be53b9f0efcad633. Latest consensus state: current_term: 1 leader_uuid: "a2dfdc6a152042c3be53b9f0efcad633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2dfdc6a152042c3be53b9f0efcad633" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36991 } } }
I20260810 06:46:00.307188 27936 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:00.307310 27936 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2dfdc6a152042c3be53b9f0efcad633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2dfdc6a152042c3be53b9f0efcad633" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36991 } } }
I20260810 06:46:00.307341 27936 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633 [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:00.307734 27943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:46:00.308462 27943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:46:00 dist-test-slave-vxj2 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:46:00.315137 27943 catalog_manager.cc:1347] Generated new cluster ID: ba0ea0821c8a48d3b31c2037161a82b3
I20260810 06:46:00.315250 27943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:46:00.319109 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.306739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52810 (local address 127.15.232.254:36991)
0810 06:46:00.307641 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:00.307655 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:00.307704 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:46:00.307865 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:00.307875 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:00.307958 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:00.308125 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:00.309262 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.310545 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.311618 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.312064 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.312152 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.315114 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:00.315119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:00.315119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:00.315133 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:00.317261 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:00.317683 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:00.317687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:00.317687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:00.317808 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:00.318037 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:00.318038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:00.318038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:00.318341 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:00.318497 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:00.318622 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 06:46:00.318868 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":101,"threads_started":1}
I20260810 06:46:00.328914 27943 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:46:00.329734 27943 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:46:00.336378 27943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633: Generated new TSK 0
I20260810 06:46:00.336613 27943 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:46:00.365046 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:36991
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:00.457129 27962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:00.457393 27962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:00.457420 27962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:00.458738 27962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:00.458786 27962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:00.458815 27962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:46:00.460493 27962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:36991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:46:00.460764 27962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:00.461028 27962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:00.466332 27968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:00.466385 27972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:00.466346 27969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:00.466588 27962 server_base.cc:1034] running on GCE node
I20260810 06:46:00.466897 27962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:00.467127 27962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:00.468303 27962 hybrid_clock.cc:648] HybridClock initialized: now 1786344360468290 us; error 29 us; skew 500 ppm
Aug 10 06:46:00 dist-test-slave-vxj2 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344360, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:00.471307 27962 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:46:00.471830 27962 webserver.cc:460] Webserver started at http://127.15.232.193:42565/ using document root <none> and password file <none>
I20260810 06:46:00.472126 27962 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:00.472180 27962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:00.472376 27962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:46:00.473644 27962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "a94104ae5c2e40478028afacb9b342dc"
format_stamp: "Formatted at 2026-08-10 06:46:00 on dist-test-slave-vxj2"
server_key: "213d4c289b4b96bd1f2f6c96e83fd55c"
server_key_iv: "eca18cb74a2710285fc0291e290fa719"
server_key_version: "encryptionkey@0"
I20260810 06:46:00.474045 27962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "a94104ae5c2e40478028afacb9b342dc"
format_stamp: "Formatted at 2026-08-10 06:46:00 on dist-test-slave-vxj2"
server_key: "213d4c289b4b96bd1f2f6c96e83fd55c"
server_key_iv: "eca18cb74a2710285fc0291e290fa719"
server_key_version: "encryptionkey@0"
I20260810 06:46:00.475574 27962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:46:00.476485 27980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:00.476686 27962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:46:00.476787 27962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "a94104ae5c2e40478028afacb9b342dc"
format_stamp: "Formatted at 2026-08-10 06:46:00 on dist-test-slave-vxj2"
server_key: "213d4c289b4b96bd1f2f6c96e83fd55c"
server_key_iv: "eca18cb74a2710285fc0291e290fa719"
server_key_version: "encryptionkey@0"
I20260810 06:46:00.476872 27962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:00.500095 27962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:00.502911 27962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:00.503104 27962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:00.503382 27962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:00.503733 27962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:00.503790 27962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:00.503825 27962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:00.503872 27962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:00.510267 27962 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:41421
I20260810 06:46:00.510365 28109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:41421 every 8 connection(s)
I20260810 06:46:00.510711 27962 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:46:00.511241 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 27962
I20260810 06:46:00.511358 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:46:00.511605 16291 external_mini_cluster.cc:1428] Setting key 0b176602b161bc97350546bcc215ff76
Aug 10 06:46:00 dist-test-slave-vxj2 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344360, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
I20260810 06:46:00.522068 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.511645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:37837 (local address 127.15.232.254:36991)
0810 06:46:00.511707 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:00.511709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:00.512011 (+ 302us) server_negotiation.cc:407] Connection header received
0810 06:46:00.512751 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:00.512752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:00.512763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:00.512813 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:00.514304 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.514833 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.515907 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.516130 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.516223 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.518796 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:00.518798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:00.518799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:00.518808 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:00.520335 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:00.520864 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:00.520865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:00.520865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:00.520922 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:00.521258 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:00.521259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:00.521259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:00.521396 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:00.521452 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:00.521893 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 06:46:00.521997 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 06:46:00.522220 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.511615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:36991 (local address 127.15.232.193:37837)
0810 06:46:00.511854 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:00.511911 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:00.512631 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:00.512909 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:00.512914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:00.513853 (+ 939us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:00.514173 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.514181 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.514946 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.514947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.515804 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.515806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.516238 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.516239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.516389 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.516981 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:00.516990 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:00.518668 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:00.520429 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:00.520437 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:00.520440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:00.520784 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:00.520996 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:00.520997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:00.520997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:00.521198 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:00.521488 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:00.521495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:00.521630 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 06:46:00.521788 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 06:46:00.521992 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":106,"threads_started":1}
I20260810 06:46:00.522728 28111 heartbeater.cc:347] Connected to a master server at 127.15.232.254:36991
I20260810 06:46:00.522821 28111 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:00.522984 28111 heartbeater.cc:511] Master 127.15.232.254:36991 requested a full tablet report, sending...
I20260810 06:46:00.523550 27893 ts_manager.cc:194] Registered new tserver with Master: a94104ae5c2e40478028afacb9b342dc (127.15.232.193:41421)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
I20260810 06:46:00.524392 27893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:37837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:46:00.541273 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:36991
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:00.636462 28117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:00.636715 28117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:00.636752 28117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:00.638121 28117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:00.638175 28117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:00.638206 28117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:46:00.639879 28117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:36991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:46:00.640102 28117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:00.640388 28117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:00.645932 28122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:00.646106 28117 server_base.cc:1034] running on GCE node
W20260810 06:46:00.646018 28126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:00.645932 28123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:00.646423 28117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:00.646701 28117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:00.647869 28117 hybrid_clock.cc:648] HybridClock initialized: now 1786344360647880 us; error 72 us; skew 500 ppm
Aug 10 06:46:00 dist-test-slave-vxj2 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344360, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:00.650677 28117 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:46:00.651238 28117 webserver.cc:460] Webserver started at http://127.15.232.194:45873/ using document root <none> and password file <none>
I20260810 06:46:00.651484 28117 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:00.651554 28117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:00.651714 28117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:46:00.652802 28117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "a238307d87dc4dbd8c7eab86a7adcb78"
format_stamp: "Formatted at 2026-08-10 06:46:00 on dist-test-slave-vxj2"
server_key: "ec39550149c15ce4b772f6cc8f939caf"
server_key_iv: "c3276d9680e30f85fb6e8863d34ef490"
server_key_version: "encryptionkey@0"
I20260810 06:46:00.653258 28117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "a238307d87dc4dbd8c7eab86a7adcb78"
format_stamp: "Formatted at 2026-08-10 06:46:00 on dist-test-slave-vxj2"
server_key: "ec39550149c15ce4b772f6cc8f939caf"
server_key_iv: "c3276d9680e30f85fb6e8863d34ef490"
server_key_version: "encryptionkey@0"
I20260810 06:46:00.654767 28117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:46:00.655632 28133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:00.655833 28117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:46:00.655995 28117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "a238307d87dc4dbd8c7eab86a7adcb78"
format_stamp: "Formatted at 2026-08-10 06:46:00 on dist-test-slave-vxj2"
server_key: "ec39550149c15ce4b772f6cc8f939caf"
server_key_iv: "c3276d9680e30f85fb6e8863d34ef490"
server_key_version: "encryptionkey@0"
I20260810 06:46:00.656085 28117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:00.679299 28117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:00.682145 28117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:00.682338 28117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:00.682616 28117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:00.682977 28117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:00.683036 28117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:00.683073 28117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:00.683125 28117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:00.689497 28117 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:46681
I20260810 06:46:00.689584 28266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:46681 every 8 connection(s)
I20260810 06:46:00.689977 28117 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:46:00 dist-test-slave-vxj2 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344360, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:46:00.698076 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 28117
I20260810 06:46:00.698195 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:46:00.698395 16291 external_mini_cluster.cc:1428] Setting key c6137f2b63eb76ce9d58dce6a5b9b685
I20260810 06:46:00.700568 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.690879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:56639 (local address 127.15.232.254:36991)
0810 06:46:00.690955 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:00.690958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:00.691077 (+ 119us) server_negotiation.cc:407] Connection header received
0810 06:46:00.691616 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:00.691617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:00.691628 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:00.691692 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:00.692762 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.693360 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.694424 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.694617 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.694675 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.697114 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:00.697116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:00.697117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:00.697124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:00.699006 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:00.699453 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:00.699454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:00.699455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:00.699525 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:00.699791 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:00.699792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:00.699793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:00.699978 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:00.700060 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:00.700379 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 06:46:00.700471 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:46:00.700804 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.690769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:36991 (local address 127.15.232.194:56639)
0810 06:46:00.690949 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:00.690994 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:00.691513 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:00.691775 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:00.691778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:00.692403 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:00.692644 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.692649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.693470 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.693472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.694331 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.694333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.694701 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.694702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.694819 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.695463 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:00.695472 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:00.696950 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:00.699081 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:00.699090 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:00.699093 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:00.699364 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:00.699603 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:00.699604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:00.699604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:00.699720 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:00.700063 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:00.700071 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:00.700223 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 06:46:00.700354 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 06:46:00.700536 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260810 06:46:00.701427 28268 heartbeater.cc:347] Connected to a master server at 127.15.232.254:36991
I20260810 06:46:00.701551 28268 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:00.701768 28268 heartbeater.cc:511] Master 127.15.232.254:36991 requested a full tablet report, sending...
I20260810 06:46:00.702291 27893 ts_manager.cc:194] Registered new tserver with Master: a238307d87dc4dbd8c7eab86a7adcb78 (127.15.232.194:46681)
I20260810 06:46:00.702636 27893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:56639
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:46:00.733749 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:36991
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:00.819243 28274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:00.819469 28274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:00.819494 28274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:00.820670 28274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:00.820715 28274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:00.820771 28274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:46:00.822276 28274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:36991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:46:00.822535 28274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:00.822813 28274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:00.828241 28280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:00.828238 28283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:00.828243 28281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:00.828521 28274 server_base.cc:1034] running on GCE node
I20260810 06:46:00.828789 28274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:00.829058 28274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:00.830228 28274 hybrid_clock.cc:648] HybridClock initialized: now 1786344360830231 us; error 46 us; skew 500 ppm
Aug 10 06:46:00 dist-test-slave-vxj2 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344360, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:00.832918 28274 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:46:00.833427 28274 webserver.cc:460] Webserver started at http://127.15.232.195:44093/ using document root <none> and password file <none>
I20260810 06:46:00.833616 28274 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:00.833657 28274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:00.833750 28274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:46:00.834787 28274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "35c3230195b248558014479dd7e7c911"
format_stamp: "Formatted at 2026-08-10 06:46:00 on dist-test-slave-vxj2"
server_key: "5744f34042a45b7cf5c73dfc44f6c32b"
server_key_iv: "c588752bf658bb51daa16f002330eb62"
server_key_version: "encryptionkey@0"
I20260810 06:46:00.835098 28274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "35c3230195b248558014479dd7e7c911"
format_stamp: "Formatted at 2026-08-10 06:46:00 on dist-test-slave-vxj2"
server_key: "5744f34042a45b7cf5c73dfc44f6c32b"
server_key_iv: "c588752bf658bb51daa16f002330eb62"
server_key_version: "encryptionkey@0"
I20260810 06:46:00.836503 28274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:46:00.837316 28291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:00.837486 28274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:46:00.837589 28274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "35c3230195b248558014479dd7e7c911"
format_stamp: "Formatted at 2026-08-10 06:46:00 on dist-test-slave-vxj2"
server_key: "5744f34042a45b7cf5c73dfc44f6c32b"
server_key_iv: "c588752bf658bb51daa16f002330eb62"
server_key_version: "encryptionkey@0"
I20260810 06:46:00.837683 28274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:00.869494 28274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:00.871771 28274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:00.871961 28274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:00.872231 28274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:00.872587 28274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:00.872645 28274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:00.872689 28274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:00.872717 28274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:00.878712 28274 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:36605
I20260810 06:46:00.878876 28424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:36605 every 8 connection(s)
I20260810 06:46:00.879170 28274 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:46:00.879530 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 28274
I20260810 06:46:00.879650 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:46:00.879871 16291 external_mini_cluster.cc:1428] Setting key 7d6ed96a688e7156dfed17d66edce901
Aug 10 06:46:00 dist-test-slave-vxj2 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344360, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:46:00.890023 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.880378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:49953 (local address 127.15.232.254:36991)
0810 06:46:00.880469 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:00.880471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:00.880484 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:46:00.881306 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:00.881307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:00.881318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:00.881359 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:00.882461 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.882955 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.884111 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.884302 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.884362 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.887051 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:00.887052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:00.887053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:00.887061 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:00.888379 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:00.888921 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:00.888923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:00.888923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:00.888984 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:00.889363 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:00.889364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:00.889365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:00.889499 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:00.889557 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:00.889878 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 06:46:00.889953 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:46:00.890161 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.880087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:36991 (local address 127.15.232.195:49953)
0810 06:46:00.880268 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:00.880308 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:00.881135 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:00.881430 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:00.881433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:00.882113 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:00.882359 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.882364 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.883056 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.883057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.884024 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.884026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.884394 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.884395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.884515 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.885251 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:00.885260 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:00.886940 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:00.888492 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:00.888499 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:00.888502 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:00.888831 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:00.889085 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:00.889086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:00.889086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:00.889291 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:00.889565 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:00.889571 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:00.889706 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 06:46:00.889793 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:46:00.889943 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260810 06:46:00.890677 28425 heartbeater.cc:347] Connected to a master server at 127.15.232.254:36991
I20260810 06:46:00.890772 28425 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:00.891003 28425 heartbeater.cc:511] Master 127.15.232.254:36991 requested a full tablet report, sending...
I20260810 06:46:00.891487 27893 ts_manager.cc:194] Registered new tserver with Master: 35c3230195b248558014479dd7e7c911 (127.15.232.195:36605)
I20260810 06:46:00.892066 27893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:49953
I20260810 06:46:00.892218 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:46:00.900400 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.893626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52818 (local address 127.15.232.254:36991)
0810 06:46:00.893728 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:00.893731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:00.893837 (+ 106us) server_negotiation.cc:407] Connection header received
0810 06:46:00.893913 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:00.893914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:00.893924 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:00.894011 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:00.894640 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.895166 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.895957 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.896148 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.896228 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.897689 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:00.897691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:00.897692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:00.897700 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:00.899080 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:00.899449 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:00.899450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:00.899450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:00.899517 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:00.899718 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:00.899719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:00.899720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:00.899884 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:00.900045 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:00.900188 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 06:46:00.900288 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:46:00.902514 27893 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:46:00.904207 27893 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:46:00.911546 28445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.907212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36568 (local address 127.15.232.193:41421)
0810 06:46:00.907368 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:00.907372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:00.907384 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:46:00.907458 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:00.907460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:00.907479 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:00.907584 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:00.907859 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.909063 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.910531 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.911089 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.911199 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.911264 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:00.911400 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 06:46:00.911441 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260810 06:46:00.911525 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.906751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:41421 (local address 127.0.0.1:36568)
0810 06:46:00.907314 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:00.907324 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:00.907388 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:00.907622 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:00.907626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:00.907629 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:00.907712 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.907715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.909184 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.909185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.910422 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.910425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.911202 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.911203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.911285 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.911299 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 06:46:00.911374 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:46:00.911436 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":42,"thread_start_us":115,"threads_started":1}
I20260810 06:46:00.911870 28442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.906731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:46681 (local address 127.0.0.1:55346)
0810 06:46:00.907057 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:00.907107 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:00.907205 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:00.907913 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:00.907914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:00.907917 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:00.907978 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.907981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.909018 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.909019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.910574 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.910576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.911452 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.911453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.911521 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.911526 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:46:00.911580 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 06:46:00.911780 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":31,"thread_start_us":66,"threads_started":1}
I20260810 06:46:00.911984 28444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.907032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49280 (local address 127.15.232.195:36605)
0810 06:46:00.907218 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:00.907223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:00.907758 (+ 535us) server_negotiation.cc:407] Connection header received
0810 06:46:00.907892 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:00.907895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:00.907912 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:00.907992 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:00.908157 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.909250 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.910421 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.911138 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.911225 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.911279 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:00.911834 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 06:46:00.911897 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":87,"threads_started":1}
I20260810 06:46:00.912083 28443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.906883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.15.232.194:46681)
0810 06:46:00.907519 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:00.907526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:00.907536 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:46:00.907572 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:00.907574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:00.907587 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:00.908058 (+ 471us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:00.908133 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.908929 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.910661 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.911360 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.911452 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.911504 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:00.911592 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 06:46:00.911634 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":105,"threads_started":1}
I20260810 06:46:00.912213 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.906771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:36605 (local address 127.0.0.1:49280)
0810 06:46:00.907680 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:00.907688 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:00.907810 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:00.908006 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:00.908007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:00.908009 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:00.908057 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.908059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.909351 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.909351 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.910336 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.910338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.911660 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.911661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.911726 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.911730 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:46:00.912095 (+ 365us) client_negotiation.cc:240] Negotiation successful
0810 06:46:00.912142 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":100,"threads_started":1}
I20260810 06:46:00.912384 28033 tablet_service.cc:1467] Processing CreateTablet for tablet 609601bdb2684259ac01f1ef612f23ba (DEFAULT_TABLE table=test-table [id=425bbe6f90ea43688b79ed0598dfefef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:00.912802 28033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 609601bdb2684259ac01f1ef612f23ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:00.912814 28188 tablet_service.cc:1467] Processing CreateTablet for tablet 609601bdb2684259ac01f1ef612f23ba (DEFAULT_TABLE table=test-table [id=425bbe6f90ea43688b79ed0598dfefef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:00.913215 28188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 609601bdb2684259ac01f1ef612f23ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:00.915850 28447 tablet_bootstrap.cc:492] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78: Bootstrap starting.
I20260810 06:46:00.916263 28348 tablet_service.cc:1467] Processing CreateTablet for tablet 609601bdb2684259ac01f1ef612f23ba (DEFAULT_TABLE table=test-table [id=425bbe6f90ea43688b79ed0598dfefef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:00.916579 28447 tablet_bootstrap.cc:654] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:00.916589 28348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 609601bdb2684259ac01f1ef612f23ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:00.917075 28447 log.cc:824] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:00.917349 28446 tablet_bootstrap.cc:492] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc: Bootstrap starting.
I20260810 06:46:00.918011 28447 tablet_bootstrap.cc:492] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78: No bootstrap required, opened a new log
I20260810 06:46:00.918020 28446 tablet_bootstrap.cc:654] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:00.918093 28447 ts_tablet_manager.cc:1397] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:46:00.918612 28446 log.cc:824] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:00.919552 28446 tablet_bootstrap.cc:492] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc: No bootstrap required, opened a new log
I20260810 06:46:00.919628 28446 ts_tablet_manager.cc:1397] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:46:00.919827 28450 tablet_bootstrap.cc:492] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911: Bootstrap starting.
I20260810 06:46:00.920279 28447 raft_consensus.cc:348] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238307d87dc4dbd8c7eab86a7adcb78" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46681 } } peers { permanent_uuid: "a94104ae5c2e40478028afacb9b342dc" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 41421 } } peers { permanent_uuid: "35c3230195b248558014479dd7e7c911" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36605 } }
I20260810 06:46:00.920428 28450 tablet_bootstrap.cc:654] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:00.920465 28447 raft_consensus.cc:374] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:00.920507 28447 raft_consensus.cc:729] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a238307d87dc4dbd8c7eab86a7adcb78, State: Initialized, Role: FOLLOWER
I20260810 06:46:00.920632 28447 consensus_queue.cc:260] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238307d87dc4dbd8c7eab86a7adcb78" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46681 } } peers { permanent_uuid: "a94104ae5c2e40478028afacb9b342dc" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 41421 } } peers { permanent_uuid: "35c3230195b248558014479dd7e7c911" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36605 } }
I20260810 06:46:00.920928 28447 ts_tablet_manager.cc:1428] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:46:00.920955 28268 heartbeater.cc:503] Master 127.15.232.254:36991 was elected leader, sending a full tablet report...
I20260810 06:46:00.921043 28450 log.cc:824] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:00.921816 28450 tablet_bootstrap.cc:492] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911: No bootstrap required, opened a new log
I20260810 06:46:00.921888 28450 ts_tablet_manager.cc:1397] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:46:00.922238 28446 raft_consensus.cc:348] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238307d87dc4dbd8c7eab86a7adcb78" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46681 } } peers { permanent_uuid: "a94104ae5c2e40478028afacb9b342dc" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 41421 } } peers { permanent_uuid: "35c3230195b248558014479dd7e7c911" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36605 } }
I20260810 06:46:00.922423 28446 raft_consensus.cc:374] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:00.922459 28446 raft_consensus.cc:729] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94104ae5c2e40478028afacb9b342dc, State: Initialized, Role: FOLLOWER
I20260810 06:46:00.922591 28446 consensus_queue.cc:260] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238307d87dc4dbd8c7eab86a7adcb78" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46681 } } peers { permanent_uuid: "a94104ae5c2e40478028afacb9b342dc" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 41421 } } peers { permanent_uuid: "35c3230195b248558014479dd7e7c911" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36605 } }
I20260810 06:46:00.922876 28446 ts_tablet_manager.cc:1428] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:46:00.922924 28111 heartbeater.cc:503] Master 127.15.232.254:36991 was elected leader, sending a full tablet report...
I20260810 06:46:00.923885 28450 raft_consensus.cc:348] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238307d87dc4dbd8c7eab86a7adcb78" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46681 } } peers { permanent_uuid: "a94104ae5c2e40478028afacb9b342dc" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 41421 } } peers { permanent_uuid: "35c3230195b248558014479dd7e7c911" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36605 } }
I20260810 06:46:00.924080 28450 raft_consensus.cc:374] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:00.924127 28450 raft_consensus.cc:729] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c3230195b248558014479dd7e7c911, State: Initialized, Role: FOLLOWER
I20260810 06:46:00.924238 28450 consensus_queue.cc:260] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238307d87dc4dbd8c7eab86a7adcb78" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46681 } } peers { permanent_uuid: "a94104ae5c2e40478028afacb9b342dc" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 41421 } } peers { permanent_uuid: "35c3230195b248558014479dd7e7c911" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36605 } }
I20260810 06:46:00.924510 28450 ts_tablet_manager.cc:1428] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:46:00.924557 28425 heartbeater.cc:503] Master 127.15.232.254:36991 was elected leader, sending a full tablet report...
W20260810 06:46:00.940614 28269 tablet.cc:2369] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:46:00.941020 28453 raft_consensus.cc:482] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:46:00.941149 28453 raft_consensus.cc:504] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238307d87dc4dbd8c7eab86a7adcb78" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46681 } } peers { permanent_uuid: "a94104ae5c2e40478028afacb9b342dc" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 41421 } } peers { permanent_uuid: "35c3230195b248558014479dd7e7c911" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36605 } }
I20260810 06:46:00.941607 28453 leader_election.cc:288] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a238307d87dc4dbd8c7eab86a7adcb78 (127.15.232.194:46681), 35c3230195b248558014479dd7e7c911 (127.15.232.195:36605)
I20260810 06:46:00.944859 28443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.941896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:34519 (local address 127.15.232.194:46681)
0810 06:46:00.941955 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:00.941957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:00.942138 (+ 181us) server_negotiation.cc:407] Connection header received
0810 06:46:00.942222 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:00.942223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:00.942232 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:00.942279 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:00.942441 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.942905 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.944121 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.944500 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.944588 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.944597 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:00.944745 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 06:46:00.944784 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 06:46:00.944869 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.941861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:46681 (local address 127.15.232.193:34519)
0810 06:46:00.942051 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:00.942062 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:00.942155 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:00.942280 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:00.942280 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:00.942282 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:00.942373 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.942375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.942990 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.942991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.944032 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.944035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.944585 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.944586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.944657 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.944661 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:46:00.944738 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 06:46:00.944778 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":1,"thread_start_us":132,"threads_started":1}
I20260810 06:46:00.944921 28444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.942034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:38589 (local address 127.15.232.195:36605)
0810 06:46:00.942089 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:00.942091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:00.942099 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:46:00.942122 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:00.942123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:00.942130 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:00.942201 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:00.942385 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.942875 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.944116 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.944510 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.944604 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.944613 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:00.944812 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 06:46:00.944853 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 06:46:00.944950 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.941806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:36605 (local address 127.15.232.193:38589)
0810 06:46:00.941868 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:00.941883 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:00.941932 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:00.942206 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:00.942208 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:00.942209 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:00.942288 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.942290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.942961 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.942962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.944022 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:00.944024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.944605 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.944606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:00.944687 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.944696 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:46:00.944796 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:46:00.944840 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":3}
I20260810 06:46:00.945379 28210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609601bdb2684259ac01f1ef612f23ba" candidate_uuid: "a94104ae5c2e40478028afacb9b342dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a238307d87dc4dbd8c7eab86a7adcb78" is_pre_election: true
I20260810 06:46:00.945403 28371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609601bdb2684259ac01f1ef612f23ba" candidate_uuid: "a94104ae5c2e40478028afacb9b342dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c3230195b248558014479dd7e7c911" is_pre_election: true
I20260810 06:46:00.945567 28210 raft_consensus.cc:2454] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a94104ae5c2e40478028afacb9b342dc in term 0.
I20260810 06:46:00.945567 28371 raft_consensus.cc:2454] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a94104ae5c2e40478028afacb9b342dc in term 0.
I20260810 06:46:00.945793 27985 leader_election.cc:302] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a238307d87dc4dbd8c7eab86a7adcb78, a94104ae5c2e40478028afacb9b342dc; no voters:
I20260810 06:46:00.945974 28453 raft_consensus.cc:2781] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:46:00.946041 28453 raft_consensus.cc:482] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:46:00.946065 28453 raft_consensus.cc:3037] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:00.946802 28453 raft_consensus.cc:504] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238307d87dc4dbd8c7eab86a7adcb78" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46681 } } peers { permanent_uuid: "a94104ae5c2e40478028afacb9b342dc" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 41421 } } peers { permanent_uuid: "35c3230195b248558014479dd7e7c911" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36605 } }
I20260810 06:46:00.946965 28453 leader_election.cc:288] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [CANDIDATE]: Term 1 election: Requested vote from peers a238307d87dc4dbd8c7eab86a7adcb78 (127.15.232.194:46681), 35c3230195b248558014479dd7e7c911 (127.15.232.195:36605)
I20260810 06:46:00.947111 28210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609601bdb2684259ac01f1ef612f23ba" candidate_uuid: "a94104ae5c2e40478028afacb9b342dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a238307d87dc4dbd8c7eab86a7adcb78"
I20260810 06:46:00.947121 28371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609601bdb2684259ac01f1ef612f23ba" candidate_uuid: "a94104ae5c2e40478028afacb9b342dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c3230195b248558014479dd7e7c911"
I20260810 06:46:00.947192 28210 raft_consensus.cc:3037] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:00.947237 28371 raft_consensus.cc:3037] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:00.947968 28210 raft_consensus.cc:2454] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a94104ae5c2e40478028afacb9b342dc in term 1.
I20260810 06:46:00.948134 27985 leader_election.cc:302] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a238307d87dc4dbd8c7eab86a7adcb78, a94104ae5c2e40478028afacb9b342dc; no voters:
I20260810 06:46:00.948140 28371 raft_consensus.cc:2454] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a94104ae5c2e40478028afacb9b342dc in term 1.
I20260810 06:46:00.948242 28453 raft_consensus.cc:2781] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:46:00.948431 28453 raft_consensus.cc:686] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 1 LEADER]: Becoming Leader. State: Replica: a94104ae5c2e40478028afacb9b342dc, State: Running, Role: LEADER
I20260810 06:46:00.948541 28453 consensus_queue.cc:237] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238307d87dc4dbd8c7eab86a7adcb78" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46681 } } peers { permanent_uuid: "a94104ae5c2e40478028afacb9b342dc" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 41421 } } peers { permanent_uuid: "35c3230195b248558014479dd7e7c911" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36605 } }
I20260810 06:46:00.949453 27892 catalog_manager.cc:5495] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc reported cstate change: term changed from 0 to 1, leader changed from <none> to a94104ae5c2e40478028afacb9b342dc (127.15.232.193). New cstate: current_term: 1 leader_uuid: "a94104ae5c2e40478028afacb9b342dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238307d87dc4dbd8c7eab86a7adcb78" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a94104ae5c2e40478028afacb9b342dc" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 41421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35c3230195b248558014479dd7e7c911" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36605 } health_report { overall_health: UNKNOWN } } }
I20260810 06:46:00.957914 28445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.955325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.15.232.193:41421)
0810 06:46:00.955375 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:00.955377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:00.955431 (+ 54us) server_negotiation.cc:407] Connection header received
0810 06:46:00.955469 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:00.955470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:00.955497 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:00.955554 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:46:00.955680 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.956206 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.957002 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:00.957217 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:00.957272 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:00.957496 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:46:00.957649 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:46:00.957753 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:00.957770 (+ 17us) server_negotiation.cc:299] Negotiation successful
0810 06:46:00.957792 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:46:00.960131 28371 raft_consensus.cc:1261] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911 [term 1 FOLLOWER]: Refusing update from remote peer a94104ae5c2e40478028afacb9b342dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:46:00.960237 28210 raft_consensus.cc:1261] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78 [term 1 FOLLOWER]: Refusing update from remote peer a94104ae5c2e40478028afacb9b342dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:46:00.960534 28456 consensus_queue.cc:1035] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "35c3230195b248558014479dd7e7c911" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 36605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:46:00.960659 28453 consensus_queue.cc:1035] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a238307d87dc4dbd8c7eab86a7adcb78" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:46:00.963022 28460 mvcc.cc:205] Tried to move back new op lower bound from 7316866502488989696 to 7316866502445785088. Current Snapshot: MvccSnapshot[applied={T|T < 7316866502488989696}]
I20260810 06:46:00.963997 28461 mvcc.cc:205] Tried to move back new op lower bound from 7316866502488989696 to 7316866502445785088. Current Snapshot: MvccSnapshot[applied={T|T < 7316866502488989696}]
I20260810 06:46:00.965755 27893 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:46:00.965860 27893 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:46:00.966696 27893 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2dfdc6a152042c3be53b9f0efcad633: Sending DeleteTablet for 3 replicas of tablet 609601bdb2684259ac01f1ef612f23ba
I20260810 06:46:00.967034 28033 tablet_service.cc:1514] Processing DeleteTablet for tablet 609601bdb2684259ac01f1ef612f23ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:00 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:36568
I20260810 06:46:00.967042 28188 tablet_service.cc:1514] Processing DeleteTablet for tablet 609601bdb2684259ac01f1ef612f23ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:00 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:55346
I20260810 06:46:00.967135 28348 tablet_service.cc:1514] Processing DeleteTablet for tablet 609601bdb2684259ac01f1ef612f23ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:00 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:49280
I20260810 06:46:00.967314 28472 tablet_replica.cc:330] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78: stopping tablet replica
I20260810 06:46:00.967427 28472 raft_consensus.cc:2229] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:46:00.967563 28472 raft_consensus.cc:2258] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:00.967689 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 27962
I20260810 06:46:00.967751 28471 tablet_replica.cc:330] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc: stopping tablet replica
I20260810 06:46:00.967847 28471 raft_consensus.cc:2229] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:46:00.967954 28471 raft_consensus.cc:2258] T 609601bdb2684259ac01f1ef612f23ba P a94104ae5c2e40478028afacb9b342dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:00.968325 28473 tablet_replica.cc:330] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911: stopping tablet replica
I20260810 06:46:00.968449 28473 raft_consensus.cc:2229] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:46:00.968577 28473 raft_consensus.cc:2258] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:00.968813 28472 ts_tablet_manager.cc:1905] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:46:00.970742 28472 ts_tablet_manager.cc:1918] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:46:00.970826 28472 log.cc:1196] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/609601bdb2684259ac01f1ef612f23ba
I20260810 06:46:00.970968 28473 ts_tablet_manager.cc:1905] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:46:00.971081 28472 ts_tablet_manager.cc:1939] T 609601bdb2684259ac01f1ef612f23ba P a238307d87dc4dbd8c7eab86a7adcb78: Deleting consensus metadata
I20260810 06:46:00.971513 27877 catalog_manager.cc:4841] TS a238307d87dc4dbd8c7eab86a7adcb78 (127.15.232.194:46681): tablet 609601bdb2684259ac01f1ef612f23ba (table test-table [id=425bbe6f90ea43688b79ed0598dfefef]) successfully deleted
I20260810 06:46:00.973457 28473 ts_tablet_manager.cc:1918] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:46:00.973654 28473 log.cc:1196] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/609601bdb2684259ac01f1ef612f23ba
I20260810 06:46:00.974068 28473 ts_tablet_manager.cc:1939] T 609601bdb2684259ac01f1ef612f23ba P 35c3230195b248558014479dd7e7c911: Deleting consensus metadata
I20260810 06:46:00.974701 27876 catalog_manager.cc:4841] TS 35c3230195b248558014479dd7e7c911 (127.15.232.195:36605): tablet 609601bdb2684259ac01f1ef612f23ba (table test-table [id=425bbe6f90ea43688b79ed0598dfefef]) successfully deleted
W20260810 06:46:00.977959 27874 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:41421 (error 108)
I20260810 06:46:00.978245 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 28117
I20260810 06:46:00.979574 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:00.978411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:41421 (local address 127.0.0.1:36590)
0810 06:46:00.979479 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:00.979530 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:41421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1041}
W20260810 06:46:00.980073 27874 catalog_manager.cc:4584] TS a94104ae5c2e40478028afacb9b342dc (127.15.232.193:41421): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 609601bdb2684259ac01f1ef612f23ba: Network error: Client connection negotiation failed: client connection to 127.15.232.193:41421: connect: Connection refused (error 111)
I20260810 06:46:00.993373 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 28274
I20260810 06:46:01.000980 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 27855
2026-08-10T06:46:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2122 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:46:01 dist-test-slave-vxj2 krb5kdc[28478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:46:01 dist-test-slave-vxj2 krb5kdc[28478](info): set up 2 sockets
Aug 10 06:46:01 dist-test-slave-vxj2 krb5kdc[28478](info): commencing operation
krb5kdc: starting...
W20260810 06:46:02.141067 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:46:02 dist-test-slave-vxj2 krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:46:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:46:02Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.232.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.232.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/rangeradmin_127.15.232.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/rangeradmin_127.15.232.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.232.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.232.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/rangerlookup_127.15.232.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/rangerlookup_127.15.232.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.232.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.232.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/HTTP_127.15.232.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/HTTP_127.15.232.212@KRBTEST.COM.keytab.
I20260810 06:46:02.341045 16291 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktpUSKt/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/postgres -l logfile start
2026-08-10 06:46:05.279 UTC [28510] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 06:46:05.279 UTC [28510] LOG: listening on IPv4 address "127.15.232.212", port 60609
2026-08-10 06:46:05.284 UTC [28510] LOG: listening on Unix socket "/tmp/.s.PGSQL.60609"
2026-08-10 06:46:05.325 UTC [28513] LOG: database system was shut down at 2026-08-10 06:46:03 UTC
2026-08-10 06:46:05.330 UTC [28510] LOG: database system is ready to accept connections
I20260810 06:46:06.379428 16291 mini_postgres.cc:96] Postgres bound to 60609
2026-08-10 06:46:06.387 UTC [28523] FATAL: database "slave" does not exist
127.15.232.212:60609 - accepting connections
I20260810 06:46:06.388826 16291 mini_ranger.cc:161] Starting Ranger...
I20260810 06:46:06.407083 16291 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 06:46:06.984843 16291 mini_ranger.cc:87] Created ranger Postgres database
I20260810 06:46:06.984937 16291 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-admin
2026-08-10 06:46:07,322 [I] DB FLAVOR :POSTGRES
2026-08-10 06:46:07,323 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:46:07,323 [I] Checking connection..
2026-08-10 06:46:07,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 06:46:07,608 [I] Checking connection passed.
2026-08-10 06:46:07,608 [I] --------- Verifying version history table ---------
2026-08-10 06:46:07,608 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:46:07,884 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:46:07,884 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:46:08,175 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:46:08,176 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 06:46:08,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 06:46:08.450 UTC [28597] WARNING: there is no transaction in progress
2026-08-10 06:46:08,459 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 06:46:08,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:46:08,744 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 06:46:08,744 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:46:08,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 06:46:09,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 06:46:09,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vxj2.c.gcp-upstream.internal','N') ;"
2026-08-10 06:46:09,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:46:09,853 [I] Table x_portal_user does not exist in database ranger
2026-08-10 06:46:09,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:46:10,144 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 06:46:10,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:46:10,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:46:10,690 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 06:46:10,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 06:46:11.797 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:11.815 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:11.836 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:11.854 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:11.876 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:11.898 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:11.966 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:11.974 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:11.991 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:12.004 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:12.016 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:12.032 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:12.043 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:12.051 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:12.059 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:12.067 UTC [28726] WARNING: there is no transaction in progress
2026-08-10 06:46:12,850 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 06:46:12,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 06:46:13,114 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 06:46:13,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:46:13,397 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 06:46:13,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:46:13,683 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 06:46:13,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:46:13,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:46:14,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vxj2.c.gcp-upstream.internal';"
2026-08-10 06:46:14,502 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 06:46:14,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 06:46:14,772 [I] DB_PATCHES have already been applied
I20260810 06:46:14.777998 16291 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktpUSKt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 06:46:14.778096 16291 mini_ranger.cc:193] Using host: 127.15.232.212
I20260810 06:46:14.781621 16291 mini_ranger.cc:227] Ranger admin URL: http://127.15.232.212:42163
Aug 10, 2026 6:46:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:46:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:46:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:46:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:46:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:46:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:46:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 06:46:16.083248 16291 mini_ranger.cc:160] Time spent starting Ranger: real 9.694s user 0.005s sys 0.006s
Aug 10, 2026 6:46:38 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:46:38 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 6:46:38 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:46:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:46:38 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:46:39 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 06:46:39.785411 16291 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.232.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.232.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/rangerkms_127.15.232.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/rangerkms_127.15.232.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.232.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.232.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.232.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/HTTP_127.15.232.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.232.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/HTTP_127.15.232.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 06:46:39 dist-test-slave-vxj2 krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344399, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 06:46:39.876935 16291 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 06:46:39.887406 16291 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 06:46:40.858844 16291 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 06:46:40.858935 16291 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-kms
I20260810 06:46:40.858948 16291 mini_ranger_kms.cc:227] Using postgres at 127.15.232.212:60609
2026-08-10 06:46:41,086 [I] DB FLAVOR :POSTGRES
2026-08-10 06:46:41,087 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:46:41,087 [I] Checking connection
2026-08-10 06:46:41,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 06:46:41,358 [I] connection success
2026-08-10 06:46:41,358 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 06:46:41,358 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 06:46:41,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 06:46:41,668 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 06:46:41,668 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:46:41,668 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 06:46:41,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:60609/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 06:46:41,963 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 06:46:42.541085 16291 mini_ranger_kms.cc:310] Created kms service
I20260810 06:46:42.782774 16291 mini_ranger_kms.cc:326] Created kudu user
I20260810 06:46:42.835418 16291 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 06:46:42 dist-test-slave-vxj2 krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344399, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:46:43.308595 16291 mini_ranger_kms.cc:384] Added ranger policy
I20260810 06:46:43.308725 16291 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 06:46:43.308748 16291 mini_ranger_kms.cc:240] Using host: 127.15.232.212
I20260810 06:46:43.312677 16291 mini_ranger_kms.cc:276] Ranger KMS PID: 28938
I20260810 06:46:43.312839 16291 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.232.212:53693
Aug 10, 2026 6:46:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:46:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:46:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:46:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:46:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:46:44 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 06:46:45 dist-test-slave-vxj2 krb5kdc[28478](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344405, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.232.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:45.786623 16291 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.910s user 0.016s sys 0.194s
I20260810 06:46:45.786733 16291 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 06:46:45.786809 16291 mini_ranger_kms.cc:401] 127.15.232.212:53693/kms/v1/keys
Aug 10 06:46:46 dist-test-slave-vxj2 krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344405, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.232.212@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
Aug 10 06:46:46 dist-test-slave-vxj2 krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344405, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.232.212@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
Aug 10 06:46:48 dist-test-slave-vxj2 krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:46:48.881628 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:37659
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:37659
--ranger_config_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.212:53693/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:48.967376 28992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:48.967617 28992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:48.967644 28992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:48.968871 28992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:46:48.968920 28992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:48.968937 28992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:48.968950 28992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:46:48.968962 28992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:46:48.970758 28992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.232.212:53693/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:37659
--ranger_config_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:37659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:46:48.971030 28992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:48.971280 28992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:48.977069 28997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:48.977240 29000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:48.977240 28998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:48.977280 28992 server_base.cc:1034] running on GCE node
I20260810 06:46:48.977727 28992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:48.978039 28992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:48.979203 28992 hybrid_clock.cc:648] HybridClock initialized: now 1786344408979168 us; error 39 us; skew 500 ppm
Aug 10 06:46:48 dist-test-slave-vxj2 krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344408, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:48.981863 28992 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:46:48.982367 28992 webserver.cc:460] Webserver started at http://127.15.232.254:37073/ using document root <none> and password file <none>
I20260810 06:46:48.982600 28992 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:48.982663 28992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:48.982828 28992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:46:48 dist-test-slave-vxj2 krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344408, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:46:49.041599 28992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "c4419e5cff5d4edcbbb428fcf9aaad61"
format_stamp: "Formatted at 2026-08-10 06:46:49 on dist-test-slave-vxj2"
server_key: "e961d965c422ede88963a0e0845a8aa9"
server_key_iv: "bc2f9c2e156d9534c16c10b0ca97862e"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:49.042069 28992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "c4419e5cff5d4edcbbb428fcf9aaad61"
format_stamp: "Formatted at 2026-08-10 06:46:49 on dist-test-slave-vxj2"
server_key: "e961d965c422ede88963a0e0845a8aa9"
server_key_iv: "bc2f9c2e156d9534c16c10b0ca97862e"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:49.044008 28992 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 06:46:49.057269 29007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:49.057616 28992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:46:49.057737 28992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "c4419e5cff5d4edcbbb428fcf9aaad61"
format_stamp: "Formatted at 2026-08-10 06:46:49 on dist-test-slave-vxj2"
server_key: "e961d965c422ede88963a0e0845a8aa9"
server_key_iv: "bc2f9c2e156d9534c16c10b0ca97862e"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:49.057847 28992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:49.072963 28992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:49.075615 28992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:49.075788 28992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:49.081192 28992 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:37659
I20260810 06:46:49.081315 29073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:37659 every 8 connection(s)
I20260810 06:46:49.081693 28992 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:46:49.082573 29074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:49.085497 29074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61: Bootstrap starting.
I20260810 06:46:49.086177 29074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:49.086575 29074 log.cc:824] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:49.087430 29074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61: No bootstrap required, opened a new log
I20260810 06:46:49.088999 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 28992
I20260810 06:46:49.089121 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:46:49.089380 16291 external_mini_cluster.cc:1428] Setting key c34bf34fee08c7c2a3498acaae70a083
I20260810 06:46:49.090550 29074 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4419e5cff5d4edcbbb428fcf9aaad61" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37659 } }
I20260810 06:46:49.090752 29074 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:49.090813 29074 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4419e5cff5d4edcbbb428fcf9aaad61, State: Initialized, Role: FOLLOWER
I20260810 06:46:49.090996 29074 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4419e5cff5d4edcbbb428fcf9aaad61" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37659 } }
I20260810 06:46:49.091102 29074 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:46:49.091143 29074 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:46:49.091243 29074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:49.092093 29074 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4419e5cff5d4edcbbb428fcf9aaad61" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37659 } }
I20260810 06:46:49.092242 29074 leader_election.cc:302] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4419e5cff5d4edcbbb428fcf9aaad61; no voters:
I20260810 06:46:49.092519 29074 leader_election.cc:288] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:46:49.092555 29080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:46:49.092830 29080 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [term 1 LEADER]: Becoming Leader. State: Replica: c4419e5cff5d4edcbbb428fcf9aaad61, State: Running, Role: LEADER
I20260810 06:46:49.092965 29080 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4419e5cff5d4edcbbb428fcf9aaad61" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37659 } }
I20260810 06:46:49.093034 29074 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:46:49.093407 29080 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4419e5cff5d4edcbbb428fcf9aaad61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4419e5cff5d4edcbbb428fcf9aaad61" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37659 } } }
I20260810 06:46:49.093526 29080 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:49.093714 29080 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4419e5cff5d4edcbbb428fcf9aaad61. Latest consensus state: current_term: 1 leader_uuid: "c4419e5cff5d4edcbbb428fcf9aaad61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4419e5cff5d4edcbbb428fcf9aaad61" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 37659 } } }
I20260810 06:46:49.093824 29080 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61 [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:49.094698 29085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:46:49.095376 29074 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:46:49.095857 29085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:46:49 dist-test-slave-vxj2 krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:46:49.102638 29085 catalog_manager.cc:1347] Generated new cluster ID: 034e30178a294cc1a4556bbbffcb62f6
I20260810 06:46:49.102804 29085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:46:49.105470 29079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:49.090061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.15.232.254:37659)
0810 06:46:49.090291 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:49.090304 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:49.090343 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:46:49.090511 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:49.090521 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:49.090604 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:49.090773 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:49.091727 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:49.093196 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:49.094313 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:49.094948 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:49.095063 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:49.098830 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:49.098885 (+ 55us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:49.098886 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:49.099031 (+ 145us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:49.102535 (+ 3504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:49.103186 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:49.103190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:49.103190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:49.103337 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:49.103568 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:49.103569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:49.103569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:49.103934 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:49.104115 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:49.104720 (+ 605us) server_negotiation.cc:299] Negotiation successful
0810 06:46:49.105091 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":109,"threads_started":1}
I20260810 06:46:49.120112 29085 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:46:49.121096 29085 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:46:49.128649 29085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61: Generated new TSK 0
I20260810 06:46:49.129009 29085 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:46:49.869069 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.212:53693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:37659
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:50.006860 29124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:50.007282 29124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:50.007368 29124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:50.009553 29124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:50.009680 29124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:50.009764 29124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:46:50.012512 29124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.232.212:53693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:37659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:46:50.012872 29124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:50.013247 29124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:50.019884 29132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:50.019886 29129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:50.020138 29124 server_base.cc:1034] running on GCE node
W20260810 06:46:50.020042 29130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:50.020520 29124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:50.020823 29124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:50.022029 29124 hybrid_clock.cc:648] HybridClock initialized: now 1786344410022000 us; error 48 us; skew 500 ppm
Aug 10 06:46:50 dist-test-slave-vxj2 krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:50.025259 29124 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:46:50.025808 29124 webserver.cc:460] Webserver started at http://127.15.232.193:37137/ using document root <none> and password file <none>
I20260810 06:46:50.026091 29124 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:50.026170 29124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:50.026341 29124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:46:50 dist-test-slave-vxj2 krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:46:50.043119 29124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "d0c9aeac26394b43afee7be1f379c608"
format_stamp: "Formatted at 2026-08-10 06:46:50 on dist-test-slave-vxj2"
server_key: "1c6370696ab42fe5186910d565cd3c94"
server_key_iv: "9d6813c3a91e6802b219e7d087fee00a"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:50.043630 29124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "d0c9aeac26394b43afee7be1f379c608"
format_stamp: "Formatted at 2026-08-10 06:46:50 on dist-test-slave-vxj2"
server_key: "1c6370696ab42fe5186910d565cd3c94"
server_key_iv: "9d6813c3a91e6802b219e7d087fee00a"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:50.046480 29124 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 06:46:50.068704 29139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:50.069082 29124 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.001s
I20260810 06:46:50.069309 29124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "d0c9aeac26394b43afee7be1f379c608"
format_stamp: "Formatted at 2026-08-10 06:46:50 on dist-test-slave-vxj2"
server_key: "1c6370696ab42fe5186910d565cd3c94"
server_key_iv: "9d6813c3a91e6802b219e7d087fee00a"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:50.069533 29124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:50.099241 29124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:50.102289 29124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:50.102552 29124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:50.102908 29124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:50.103394 29124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:50.103464 29124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:50.103545 29124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:50.103585 29124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:50.112980 29124 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:46863
I20260810 06:46:50.113106 29271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:46863 every 8 connection(s)
I20260810 06:46:50.113706 29124 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:46:50.114336 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 29124
I20260810 06:46:50.114460 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:46:50.114748 16291 external_mini_cluster.cc:1428] Setting key 36495a43409e05cf32433aff4fe716be
Aug 10 06:46:50 dist-test-slave-vxj2 krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
I20260810 06:46:50.128459 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.114566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:37659 (local address 127.15.232.193:54195)
0810 06:46:50.115191 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:50.115258 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:50.116064 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:50.116528 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:50.116533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:50.116853 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:50.117219 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.117227 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.118432 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.118433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.119987 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.119991 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.120744 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.120746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.120931 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.121734 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:50.121748 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:50.123720 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:50.126430 (+ 2710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:50.126440 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:50.126443 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:50.126865 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:50.127159 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:50.127160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:50.127161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:50.127316 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:50.127673 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:50.127681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:50.127937 (+ 256us) client_negotiation.cc:769] Sending connection context
0810 06:46:50.128081 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 06:46:50.128257 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":89,"threads_started":1}
I20260810 06:46:50.129230 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.114677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:54195 (local address 127.15.232.254:37659)
0810 06:46:50.115155 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:50.115158 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:50.116248 (+ 1090us) server_negotiation.cc:407] Connection header received
0810 06:46:50.116281 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:50.116283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:50.116302 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:50.116384 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:50.117365 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.118291 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.120113 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.120563 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.120650 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.123842 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:50.123844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:50.123844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:50.123854 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:50.126306 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:50.126981 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:50.126983 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:50.126984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:50.127066 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:50.127413 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:50.127415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:50.127415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:50.127568 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:50.127667 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:50.128974 (+ 1307us) server_negotiation.cc:299] Negotiation successful
0810 06:46:50.129074 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
I20260810 06:46:50.130013 29273 heartbeater.cc:347] Connected to a master server at 127.15.232.254:37659
I20260810 06:46:50.130136 29273 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:50.130374 29273 heartbeater.cc:511] Master 127.15.232.254:37659 requested a full tablet report, sending...
I20260810 06:46:50.131057 29029 ts_manager.cc:194] Registered new tserver with Master: d0c9aeac26394b43afee7be1f379c608 (127.15.232.193:46863)
I20260810 06:46:50.132054 29029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:54195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:46:50.148680 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.212:53693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:37659
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:50.275005 29279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:50.275328 29279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:50.275374 29279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:50.277480 29279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:50.277558 29279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:50.277614 29279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:46:50.280244 29279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.232.212:53693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:37659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:46:50.280524 29279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:50.280930 29279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:50.289779 29286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:50.289775 29287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:50.290380 29289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:50.291071 29279 server_base.cc:1034] running on GCE node
I20260810 06:46:50.291299 29279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:50.291599 29279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:50.292827 29279 hybrid_clock.cc:648] HybridClock initialized: now 1786344410292808 us; error 35 us; skew 500 ppm
Aug 10 06:46:50 dist-test-slave-vxj2 krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:50.296316 29279 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:46:50.296905 29279 webserver.cc:460] Webserver started at http://127.15.232.194:38857/ using document root <none> and password file <none>
I20260810 06:46:50.297238 29279 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:50.297307 29279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:50.297516 29279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:46:50 dist-test-slave-vxj2 krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
Aug 10 06:46:50 dist-test-slave-vxj2 krb5kdc[28478](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:50.326813 29279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "101847d8b0234acb9d38522d80043930"
format_stamp: "Formatted at 2026-08-10 06:46:50 on dist-test-slave-vxj2"
server_key: "baa66bb2447949e91cec0a4d9d91134f"
server_key_iv: "057bb2b58acc865ff6c1090f1c11340c"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:50.327590 29279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "101847d8b0234acb9d38522d80043930"
format_stamp: "Formatted at 2026-08-10 06:46:50 on dist-test-slave-vxj2"
server_key: "baa66bb2447949e91cec0a4d9d91134f"
server_key_iv: "057bb2b58acc865ff6c1090f1c11340c"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:50.334314 29279 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.002s
I20260810 06:46:50.371179 29297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:50.371411 29279 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.001s
I20260810 06:46:50.371526 29279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "101847d8b0234acb9d38522d80043930"
format_stamp: "Formatted at 2026-08-10 06:46:50 on dist-test-slave-vxj2"
server_key: "baa66bb2447949e91cec0a4d9d91134f"
server_key_iv: "057bb2b58acc865ff6c1090f1c11340c"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:50.371650 29279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:50.395895 29279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:50.398805 29279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:50.399061 29279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:50.399461 29279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:50.399932 29279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:50.400038 29279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:50.400113 29279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:50.400180 29279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:50.408114 29279 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:46639
I20260810 06:46:50.408646 29279 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:46:50.410847 29424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:46639 every 8 connection(s)
I20260810 06:46:50.416991 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 29279
I20260810 06:46:50.417109 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:46:50.417356 16291 external_mini_cluster.cc:1428] Setting key 908c41986e5363c336c62067b7bb3965
Aug 10 06:46:50 dist-test-slave-vxj2 krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:46:50.430665 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.412323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:58773 (local address 127.15.232.254:37659)
0810 06:46:50.412373 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:50.412376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:50.412719 (+ 343us) server_negotiation.cc:407] Connection header received
0810 06:46:50.413533 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:50.413535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:50.413551 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:50.413583 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:50.414468 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.415278 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.420272 (+ 4994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.420597 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.420649 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.423753 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:50.423755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:50.423756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:50.423775 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:50.425768 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:50.429287 (+ 3519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:50.429289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:50.429290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:50.429378 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:50.429755 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:50.429756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:50.429756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:50.429935 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:50.430010 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:50.430434 (+ 424us) server_negotiation.cc:299] Negotiation successful
0810 06:46:50.430555 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
I20260810 06:46:50.431704 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.412016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:37659 (local address 127.15.232.194:58773)
0810 06:46:50.412579 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:50.412624 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:50.413400 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:50.413791 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:50.413795 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:50.414059 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:50.414346 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.414353 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.415403 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.415404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.420147 (+ 4743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.420152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.420776 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.420777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.420942 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.421750 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:50.421761 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:50.423602 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:50.428566 (+ 4964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:50.428582 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:50.428585 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:50.429104 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:50.429505 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:50.429506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:50.429507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:50.429660 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:50.430044 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:50.430052 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:50.430252 (+ 200us) client_negotiation.cc:769] Sending connection context
0810 06:46:50.431156 (+ 904us) client_negotiation.cc:240] Negotiation successful
0810 06:46:50.431375 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":87,"threads_started":1}
I20260810 06:46:50.433571 29425 heartbeater.cc:347] Connected to a master server at 127.15.232.254:37659
I20260810 06:46:50.433701 29425 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:50.433926 29425 heartbeater.cc:511] Master 127.15.232.254:37659 requested a full tablet report, sending...
I20260810 06:46:50.434446 29029 ts_manager.cc:194] Registered new tserver with Master: 101847d8b0234acb9d38522d80043930 (127.15.232.194:46639)
I20260810 06:46:50.435115 29029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:58773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:46:50.457144 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.212:53693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:37659
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:50.585952 29432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:50.586362 29432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:50.586484 29432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:50.588543 29432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:50.588670 29432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:50.588757 29432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:46:50.591017 29432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.232.212:53693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:37659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:46:50.591470 29432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:50.591858 29432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:50.599056 29437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:50.599086 29441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:50.599305 29432 server_base.cc:1034] running on GCE node
W20260810 06:46:50.599395 29439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:50.599781 29432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:50.600106 29432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:50.601248 29432 hybrid_clock.cc:648] HybridClock initialized: now 1786344410601229 us; error 36 us; skew 500 ppm
Aug 10 06:46:50 dist-test-slave-vxj2 krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:50.604565 29432 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:46:50.605126 29432 webserver.cc:460] Webserver started at http://127.15.232.195:39235/ using document root <none> and password file <none>
I20260810 06:46:50.605459 29432 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:50.605541 29432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:50.605717 29432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:46:50 dist-test-slave-vxj2 krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:46:50.619020 29432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "b50974f0184948c0872b7c17afb11ff2"
format_stamp: "Formatted at 2026-08-10 06:46:50 on dist-test-slave-vxj2"
server_key: "b092f825c63cb9a0a3022ef1b1fd9c5d"
server_key_iv: "acc69212a2343b3049593ba797427918"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:50.619535 29432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "b50974f0184948c0872b7c17afb11ff2"
format_stamp: "Formatted at 2026-08-10 06:46:50 on dist-test-slave-vxj2"
server_key: "b092f825c63cb9a0a3022ef1b1fd9c5d"
server_key_iv: "acc69212a2343b3049593ba797427918"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:50.625638 29432 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.000s
I20260810 06:46:50.652967 29448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:50.653316 29432 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260810 06:46:50.653447 29432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "b50974f0184948c0872b7c17afb11ff2"
format_stamp: "Formatted at 2026-08-10 06:46:50 on dist-test-slave-vxj2"
server_key: "b092f825c63cb9a0a3022ef1b1fd9c5d"
server_key_iv: "acc69212a2343b3049593ba797427918"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:50.653584 29432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:50.661337 29432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:50.663753 29432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:50.663928 29432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:50.664186 29432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:50.664534 29432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:50.664593 29432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:50.664669 29432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:50.664698 29432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:50.672874 29432 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:37061
I20260810 06:46:50.673496 29432 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:46:50.682055 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 29432
I20260810 06:46:50.682209 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:46:50.682448 16291 external_mini_cluster.cc:1428] Setting key 9ab8d20fec16938a892804db9bd7b677
I20260810 06:46:50.683444 29586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:37061 every 8 connection(s)
Aug 10 06:46:50 dist-test-slave-vxj2 krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:46:50.695236 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.684539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:39045 (local address 127.15.232.254:37659)
0810 06:46:50.684585 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:50.684587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:50.685452 (+ 865us) server_negotiation.cc:407] Connection header received
0810 06:46:50.686204 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:50.686206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:50.686219 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:50.686251 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:50.687252 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.687982 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.689124 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.689382 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.689475 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.691706 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:50.691707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:50.691708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:50.691716 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:50.693346 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:50.694004 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:50.694006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:50.694006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:50.694090 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:50.694463 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:50.694464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:50.694465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:50.694646 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:50.694724 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:50.695057 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 06:46:50.695144 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:46:50.695436 29592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.684207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:37659 (local address 127.15.232.195:39045)
0810 06:46:50.685308 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:50.685364 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:50.686073 (+ 709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:50.686488 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:50.686492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:50.686798 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:50.687130 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.687137 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.688106 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.688107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.689003 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.689005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.689486 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.689488 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.689650 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.690234 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:50.690243 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:50.691604 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:50.693455 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:50.693463 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:50.693465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:50.693919 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:50.694190 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:50.694191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:50.694191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:50.694376 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:50.694743 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:50.694751 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:50.694907 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 06:46:50.695021 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 06:46:50.695177 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1042,"thread_start_us":134,"threads_started":1}
I20260810 06:46:50.696019 29587 heartbeater.cc:347] Connected to a master server at 127.15.232.254:37659
I20260810 06:46:50.696149 29587 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:50.696346 29587 heartbeater.cc:511] Master 127.15.232.254:37659 requested a full tablet report, sending...
I20260810 06:46:50.696766 29029 ts_manager.cc:194] Registered new tserver with Master: b50974f0184948c0872b7c17afb11ff2 (127.15.232.195:37061)
I20260810 06:46:50.697206 29029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:39045
I20260810 06:46:50.705071 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:46:50.714305 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.707084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46098 (local address 127.15.232.254:37659)
0810 06:46:50.707136 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:50.707139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:50.707499 (+ 360us) server_negotiation.cc:407] Connection header received
0810 06:46:50.707551 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:50.707552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:50.707563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:50.707609 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:50.708231 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.708764 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.709630 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.709834 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.709924 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.711223 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:50.711224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:50.711225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:50.711233 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:50.712774 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:50.713305 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:50.713306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:50.713307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:50.713373 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:50.713599 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:50.713600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:50.713600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:50.713735 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:50.713868 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:50.714107 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 06:46:50.714190 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:46:50.716715 29029 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:46:50.718689 29029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:46:50.726511 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.721435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46904 (local address 127.15.232.193:46863)
0810 06:46:50.721594 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:50.721599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:50.721863 (+ 264us) server_negotiation.cc:407] Connection header received
0810 06:46:50.722025 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:50.722028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:50.722046 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:50.722104 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:50.722420 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.723655 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.725228 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.725984 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.726032 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.726093 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:50.726364 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 06:46:50.726412 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":48,"threads_started":1}
I20260810 06:46:50.727087 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.721344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:46863 (local address 127.0.0.1:46904)
0810 06:46:50.721746 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:50.721794 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:50.721909 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:50.722227 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:50.722230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:50.722232 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:50.722319 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.722321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.723748 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.723748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.725075 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.725078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.726151 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.726152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.726225 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.726240 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 06:46:50.726909 (+ 669us) client_negotiation.cc:240] Negotiation successful
0810 06:46:50.726973 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":86,"threads_started":1}
I20260810 06:46:50.728087 29198 tablet_service.cc:1467] Processing CreateTablet for tablet 99b6ddfdb42d45e1923378d639df3673 (DEFAULT_TABLE table=test-table [id=8cd2f6da9aff4da187b22465d4b92409]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:50.728466 29198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b6ddfdb42d45e1923378d639df3673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:50.731343 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.721747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:37061 (local address 127.0.0.1:44892)
0810 06:46:50.727372 (+ 5625us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:50.727383 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:50.727428 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:50.727725 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:50.727726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:50.727727 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:50.727755 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.727756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.728950 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.728951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.729985 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.729988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.731097 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.731098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.731156 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.731159 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:46:50.731211 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:46:50.731248 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5612,"thread_start_us":76,"threads_started":1}
I20260810 06:46:50.731683 29609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.722299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44892 (local address 127.15.232.195:37061)
0810 06:46:50.722799 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:50.722805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:50.727503 (+ 4698us) server_negotiation.cc:407] Connection header received
0810 06:46:50.727553 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:50.727555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:50.727571 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:50.727644 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:50.727825 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.728849 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.730156 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.730907 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.731006 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.731061 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:50.731536 (+ 475us) server_negotiation.cc:299] Negotiation successful
0810 06:46:50.731580 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":87,"threads_started":1}
I20260810 06:46:50.732348 29503 tablet_service.cc:1467] Processing CreateTablet for tablet 99b6ddfdb42d45e1923378d639df3673 (DEFAULT_TABLE table=test-table [id=8cd2f6da9aff4da187b22465d4b92409]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:50.732693 29503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b6ddfdb42d45e1923378d639df3673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:50.733206 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.721584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:46639 (local address 127.0.0.1:36974)
0810 06:46:50.722866 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:50.722875 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:50.722922 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:50.723263 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:50.723263 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:50.723265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:50.723304 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.723305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.724747 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.724748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.725888 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.725891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.727309 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.727310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.732997 (+ 5687us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.733002 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:46:50.733059 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 06:46:50.733103 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1265,"thread_start_us":70,"threads_started":1}
I20260810 06:46:50.733567 29608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.722077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36974 (local address 127.15.232.194:46639)
0810 06:46:50.723046 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:50.723050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:50.723059 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:46:50.723101 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:50.723103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:50.723118 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:50.723189 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:50.723371 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.724666 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.726015 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.727117 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.727220 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.727277 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:50.733426 (+ 6149us) server_negotiation.cc:299] Negotiation successful
0810 06:46:50.733485 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":933,"thread_start_us":96,"threads_started":1}
I20260810 06:46:50.734205 29351 tablet_service.cc:1467] Processing CreateTablet for tablet 99b6ddfdb42d45e1923378d639df3673 (DEFAULT_TABLE table=test-table [id=8cd2f6da9aff4da187b22465d4b92409]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:50.734552 29351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b6ddfdb42d45e1923378d639df3673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:50.736812 29610 tablet_bootstrap.cc:492] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608: Bootstrap starting.
I20260810 06:46:50.737061 29611 tablet_bootstrap.cc:492] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2: Bootstrap starting.
I20260810 06:46:50.738160 29611 tablet_bootstrap.cc:654] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:50.738714 29612 tablet_bootstrap.cc:492] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930: Bootstrap starting.
I20260810 06:46:50.739033 29610 tablet_bootstrap.cc:654] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:50.739416 29610 log.cc:824] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:50.739920 29612 tablet_bootstrap.cc:654] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:50.739969 29611 log.cc:824] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:50.740305 29612 log.cc:824] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:50.740849 29611 tablet_bootstrap.cc:492] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2: No bootstrap required, opened a new log
I20260810 06:46:50.740919 29611 ts_tablet_manager.cc:1397] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 06:46:50.741120 29610 tablet_bootstrap.cc:492] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608: No bootstrap required, opened a new log
I20260810 06:46:50.741200 29610 ts_tablet_manager.cc:1397] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 06:46:50.743124 29611 raft_consensus.cc:348] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101847d8b0234acb9d38522d80043930" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46639 } } peers { permanent_uuid: "d0c9aeac26394b43afee7be1f379c608" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46863 } } peers { permanent_uuid: "b50974f0184948c0872b7c17afb11ff2" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 37061 } }
I20260810 06:46:50.743321 29611 raft_consensus.cc:374] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:50.743363 29611 raft_consensus.cc:729] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50974f0184948c0872b7c17afb11ff2, State: Initialized, Role: FOLLOWER
I20260810 06:46:50.743479 29611 consensus_queue.cc:260] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101847d8b0234acb9d38522d80043930" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46639 } } peers { permanent_uuid: "d0c9aeac26394b43afee7be1f379c608" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46863 } } peers { permanent_uuid: "b50974f0184948c0872b7c17afb11ff2" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 37061 } }
I20260810 06:46:50.743739 29611 ts_tablet_manager.cc:1428] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:46:50.743979 29610 raft_consensus.cc:348] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101847d8b0234acb9d38522d80043930" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46639 } } peers { permanent_uuid: "d0c9aeac26394b43afee7be1f379c608" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46863 } } peers { permanent_uuid: "b50974f0184948c0872b7c17afb11ff2" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 37061 } }
I20260810 06:46:50.744104 29587 heartbeater.cc:503] Master 127.15.232.254:37659 was elected leader, sending a full tablet report...
I20260810 06:46:50.744453 29610 raft_consensus.cc:374] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:50.744572 29610 raft_consensus.cc:729] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c9aeac26394b43afee7be1f379c608, State: Initialized, Role: FOLLOWER
I20260810 06:46:50.744756 29610 consensus_queue.cc:260] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101847d8b0234acb9d38522d80043930" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46639 } } peers { permanent_uuid: "d0c9aeac26394b43afee7be1f379c608" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46863 } } peers { permanent_uuid: "b50974f0184948c0872b7c17afb11ff2" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 37061 } }
I20260810 06:46:50.745141 29610 ts_tablet_manager.cc:1428] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260810 06:46:50.745340 29273 heartbeater.cc:503] Master 127.15.232.254:37659 was elected leader, sending a full tablet report...
I20260810 06:46:50.746495 29612 tablet_bootstrap.cc:492] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930: No bootstrap required, opened a new log
I20260810 06:46:50.746608 29612 ts_tablet_manager.cc:1397] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260810 06:46:50.748301 29612 raft_consensus.cc:348] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101847d8b0234acb9d38522d80043930" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46639 } } peers { permanent_uuid: "d0c9aeac26394b43afee7be1f379c608" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46863 } } peers { permanent_uuid: "b50974f0184948c0872b7c17afb11ff2" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 37061 } }
I20260810 06:46:50.748502 29612 raft_consensus.cc:374] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:50.748565 29612 raft_consensus.cc:729] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101847d8b0234acb9d38522d80043930, State: Initialized, Role: FOLLOWER
I20260810 06:46:50.748720 29612 consensus_queue.cc:260] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101847d8b0234acb9d38522d80043930" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46639 } } peers { permanent_uuid: "d0c9aeac26394b43afee7be1f379c608" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46863 } } peers { permanent_uuid: "b50974f0184948c0872b7c17afb11ff2" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 37061 } }
I20260810 06:46:50.749025 29425 heartbeater.cc:503] Master 127.15.232.254:37659 was elected leader, sending a full tablet report...
I20260810 06:46:50.749009 29612 ts_tablet_manager.cc:1428] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:46:50.823570 29616 raft_consensus.cc:482] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:46:50.823716 29616 raft_consensus.cc:504] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101847d8b0234acb9d38522d80043930" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46639 } } peers { permanent_uuid: "d0c9aeac26394b43afee7be1f379c608" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46863 } } peers { permanent_uuid: "b50974f0184948c0872b7c17afb11ff2" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 37061 } }
I20260810 06:46:50.824175 29616 leader_election.cc:288] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 101847d8b0234acb9d38522d80043930 (127.15.232.194:46639), d0c9aeac26394b43afee7be1f379c608 (127.15.232.193:46863)
Aug 10 06:46:50 dist-test-slave-vxj2 krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:46:50.828843 29608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.825266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:33357 (local address 127.15.232.194:46639)
0810 06:46:50.825334 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:50.825336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:50.825348 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:46:50.825381 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:50.825381 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:50.825390 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:50.825453 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:50.825681 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.826423 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.827880 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.828456 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.828504 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.828512 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:50.828728 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 06:46:50.828768 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:46:50.828866 29592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.824445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:46863 (local address 127.15.232.195:45723)
0810 06:46:50.824662 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:50.824672 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:50.824715 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:50.825559 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:50.825560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:50.825562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:50.825628 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.825629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.826544 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.826545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.827700 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.827703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.828616 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.828617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.828688 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.828692 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:46:50.828736 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 06:46:50.828792 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":114,"threads_started":1}
I20260810 06:46:50.829123 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.824554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:45723 (local address 127.15.232.193:46863)
0810 06:46:50.825349 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:50.825351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:50.825362 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:46:50.825386 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:50.825387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:50.825397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:50.825463 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:50.825721 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.826425 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.827850 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.828407 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.828470 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.828480 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:50.829020 (+ 540us) server_negotiation.cc:299] Negotiation successful
0810 06:46:50.829064 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":766}
I20260810 06:46:50.829420 29219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99b6ddfdb42d45e1923378d639df3673" candidate_uuid: "b50974f0184948c0872b7c17afb11ff2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0c9aeac26394b43afee7be1f379c608" is_pre_election: true
I20260810 06:46:50.829607 29219 raft_consensus.cc:2454] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50974f0184948c0872b7c17afb11ff2 in term 0.
I20260810 06:46:50.829862 29620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.824329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:46639 (local address 127.15.232.195:33357)
0810 06:46:50.824652 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:50.824665 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:50.824734 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:50.825513 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:50.825514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:50.825516 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:50.825600 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.825602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.826567 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.826568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.827771 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:50.827774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.828572 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.828573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:50.828651 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.828655 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:46:50.829717 (+ 1062us) client_negotiation.cc:240] Negotiation successful
0810 06:46:50.829760 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298}
I20260810 06:46:50.829881 29452 leader_election.cc:302] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50974f0184948c0872b7c17afb11ff2, d0c9aeac26394b43afee7be1f379c608; no voters:
I20260810 06:46:50.830295 29616 raft_consensus.cc:2781] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:46:50.830345 29616 raft_consensus.cc:482] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:46:50.830366 29616 raft_consensus.cc:3037] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:50.831252 29616 raft_consensus.cc:504] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101847d8b0234acb9d38522d80043930" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46639 } } peers { permanent_uuid: "d0c9aeac26394b43afee7be1f379c608" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46863 } } peers { permanent_uuid: "b50974f0184948c0872b7c17afb11ff2" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 37061 } }
I20260810 06:46:50.831389 29616 leader_election.cc:288] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [CANDIDATE]: Term 1 election: Requested vote from peers 101847d8b0234acb9d38522d80043930 (127.15.232.194:46639), d0c9aeac26394b43afee7be1f379c608 (127.15.232.193:46863)
I20260810 06:46:50.831545 29371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99b6ddfdb42d45e1923378d639df3673" candidate_uuid: "b50974f0184948c0872b7c17afb11ff2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101847d8b0234acb9d38522d80043930" is_pre_election: true
I20260810 06:46:50.831676 29371 raft_consensus.cc:2454] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50974f0184948c0872b7c17afb11ff2 in term 0.
I20260810 06:46:50.831826 29370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99b6ddfdb42d45e1923378d639df3673" candidate_uuid: "b50974f0184948c0872b7c17afb11ff2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101847d8b0234acb9d38522d80043930"
I20260810 06:46:50.831885 29370 raft_consensus.cc:3037] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:50.832695 29370 raft_consensus.cc:2454] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50974f0184948c0872b7c17afb11ff2 in term 1.
I20260810 06:46:50.832909 29451 leader_election.cc:302] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101847d8b0234acb9d38522d80043930, b50974f0184948c0872b7c17afb11ff2; no voters:
I20260810 06:46:50.833091 29219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99b6ddfdb42d45e1923378d639df3673" candidate_uuid: "b50974f0184948c0872b7c17afb11ff2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0c9aeac26394b43afee7be1f379c608"
I20260810 06:46:50.833225 29219 raft_consensus.cc:3037] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:50.834080 29219 raft_consensus.cc:2454] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50974f0184948c0872b7c17afb11ff2 in term 1.
I20260810 06:46:50.834363 29616 raft_consensus.cc:2781] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:46:50.834530 29616 raft_consensus.cc:686] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 1 LEADER]: Becoming Leader. State: Replica: b50974f0184948c0872b7c17afb11ff2, State: Running, Role: LEADER
I20260810 06:46:50.834618 29616 consensus_queue.cc:237] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101847d8b0234acb9d38522d80043930" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46639 } } peers { permanent_uuid: "d0c9aeac26394b43afee7be1f379c608" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46863 } } peers { permanent_uuid: "b50974f0184948c0872b7c17afb11ff2" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 37061 } }
I20260810 06:46:50.835666 29028 catalog_manager.cc:5495] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b50974f0184948c0872b7c17afb11ff2 (127.15.232.195). New cstate: current_term: 1 leader_uuid: "b50974f0184948c0872b7c17afb11ff2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "101847d8b0234acb9d38522d80043930" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0c9aeac26394b43afee7be1f379c608" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50974f0184948c0872b7c17afb11ff2" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 37061 } health_report { overall_health: HEALTHY } } }
I20260810 06:46:50.860881 29609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.857407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44908 (local address 127.15.232.195:37061)
0810 06:46:50.857459 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:50.857461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:50.857473 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:46:50.857526 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:50.857527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:50.857553 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:50.857642 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:46:50.857854 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.858580 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.859740 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:50.860036 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:50.860098 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:50.860298 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:46:50.860450 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:46:50.860527 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:50.860765 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 06:46:50.860786 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260810 06:46:50.864938 29274 tablet.cc:2369] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:46:50.865909 29219 raft_consensus.cc:1261] T 99b6ddfdb42d45e1923378d639df3673 P d0c9aeac26394b43afee7be1f379c608 [term 1 FOLLOWER]: Refusing update from remote peer b50974f0184948c0872b7c17afb11ff2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:46:50.866394 29616 consensus_queue.cc:1035] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0c9aeac26394b43afee7be1f379c608" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:46:50.868505 29370 raft_consensus.cc:1261] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930 [term 1 FOLLOWER]: Refusing update from remote peer b50974f0184948c0872b7c17afb11ff2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:46:50.868888 29616 consensus_queue.cc:1035] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101847d8b0234acb9d38522d80043930" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 46639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:46:50.873186 29624 mvcc.cc:205] Tried to move back new op lower bound from 7316866706897043456 to 7316866706779254784. Current Snapshot: MvccSnapshot[applied={T|T < 7316866706897043456}]
I20260810 06:46:50.873840 29626 mvcc.cc:205] Tried to move back new op lower bound from 7316866706897043456 to 7316866706779254784. Current Snapshot: MvccSnapshot[applied={T|T < 7316866706897043456}]
I20260810 06:46:50.876814 29629 mvcc.cc:205] Tried to move back new op lower bound from 7316866706897043456 to 7316866706779254784. Current Snapshot: MvccSnapshot[applied={T|T < 7316866706897043456}]
I20260810 06:46:50.877003 29028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:46:50.877101 29028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:46:50.878031 29028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4419e5cff5d4edcbbb428fcf9aaad61: Sending DeleteTablet for 3 replicas of tablet 99b6ddfdb42d45e1923378d639df3673
I20260810 06:46:50.878835 29503 tablet_service.cc:1514] Processing DeleteTablet for tablet 99b6ddfdb42d45e1923378d639df3673 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:50 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:44892
I20260810 06:46:50.879113 29636 tablet_replica.cc:330] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2: stopping tablet replica
I20260810 06:46:50.879119 29198 tablet_service.cc:1514] Processing DeleteTablet for tablet 99b6ddfdb42d45e1923378d639df3673 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:50 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:46904
I20260810 06:46:50.879218 29636 raft_consensus.cc:2229] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:46:50.879317 29636 raft_consensus.cc:2258] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:50.878667 29351 tablet_service.cc:1514] Processing DeleteTablet for tablet 99b6ddfdb42d45e1923378d639df3673 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:50 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:36974
I20260810 06:46:50.879673 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 29124
I20260810 06:46:50.885191 29636 ts_tablet_manager.cc:1905] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:46:50.887336 29636 ts_tablet_manager.cc:1918] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:46:50.887482 29636 log.cc:1196] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/99b6ddfdb42d45e1923378d639df3673
I20260810 06:46:50.887879 29636 ts_tablet_manager.cc:1939] T 99b6ddfdb42d45e1923378d639df3673 P b50974f0184948c0872b7c17afb11ff2: Deleting consensus metadata
I20260810 06:46:50.888510 29009 catalog_manager.cc:4841] TS b50974f0184948c0872b7c17afb11ff2 (127.15.232.195:37061): tablet 99b6ddfdb42d45e1923378d639df3673 (table test-table [id=8cd2f6da9aff4da187b22465d4b92409]) successfully deleted
I20260810 06:46:50.896366 29638 tablet_replica.cc:330] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930: stopping tablet replica
I20260810 06:46:50.897476 29638 raft_consensus.cc:2229] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:46:50.898633 29638 raft_consensus.cc:2258] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:50.909500 29638 ts_tablet_manager.cc:1905] T 99b6ddfdb42d45e1923378d639df3673 P 101847d8b0234acb9d38522d80043930: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 06:46:50.909744 29010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:46863 (error 108)
I20260810 06:46:50.910034 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 29279
I20260810 06:46:50.912464 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.912091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:46863 (local address 127.0.0.1:46918)
0810 06:46:50.912367 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:50.912427 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:46863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246}
W20260810 06:46:50.913132 29010 catalog_manager.cc:4584] TS d0c9aeac26394b43afee7be1f379c608 (127.15.232.193:46863): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 99b6ddfdb42d45e1923378d639df3673: Network error: Client connection negotiation failed: client connection to 127.15.232.193:46863: connect: Connection refused (error 111)
I20260810 06:46:50.918442 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:50.918334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:46639 (local address 127.0.0.1:36980)
0810 06:46:50.918384 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:50.918406 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.194:46639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 06:46:50.919027 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 29432
I20260810 06:46:50.936182 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 28992
I20260810 06:46:50.949111 16291 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 06:46:51.294337 16291 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 06:46:51.294447 16291 mini_ranger.cc:66] Stopping Ranger...
I20260810 06:46:51.639930 16291 mini_ranger.cc:68] Stopped Ranger
2026-08-10 06:46:51.640 UTC [28510] LOG: received smart shutdown request
2026-08-10 06:46:51.644 UTC [28510] LOG: background worker "logical replication launcher" (PID 28521) exited with exit code 1
2026-08-10 06:46:51.645 UTC [28514] LOG: shutting down
2026-08-10 06:46:51.837 UTC [28510] LOG: database system is shut down
2026-08-10T06:46:51Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50954 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:46:52 dist-test-slave-vxj2 krb5kdc[29649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:46:52 dist-test-slave-vxj2 krb5kdc[29649](info): set up 2 sockets
Aug 10 06:46:52 dist-test-slave-vxj2 krb5kdc[29649](info): commencing operation
krb5kdc: starting...
W20260810 06:46:53.086228 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:46:53 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:46:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:46:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.232.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.232.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/rangeradmin_127.15.232.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/rangeradmin_127.15.232.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.232.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.232.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/rangerlookup_127.15.232.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/rangerlookup_127.15.232.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.232.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.232.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/HTTP_127.15.232.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/HTTP_127.15.232.212@KRBTEST.COM.keytab.
I20260810 06:46:53.264217 16291 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktpUSKt/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/postgres -l logfile start
2026-08-10 06:46:56.011 UTC [29684] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 06:46:56.011 UTC [29684] LOG: listening on IPv4 address "127.15.232.212", port 42523
2026-08-10 06:46:56.015 UTC [29684] LOG: listening on Unix socket "/tmp/.s.PGSQL.42523"
2026-08-10 06:46:56.052 UTC [29687] LOG: database system was shut down at 2026-08-10 06:46:54 UTC
2026-08-10 06:46:56.056 UTC [29684] LOG: database system is ready to accept connections
I20260810 06:46:57.168215 16291 mini_postgres.cc:96] Postgres bound to 42523
2026-08-10 06:46:57.175 UTC [29695] FATAL: database "slave" does not exist
127.15.232.212:42523 - accepting connections
I20260810 06:46:57.175858 16291 mini_ranger.cc:161] Starting Ranger...
I20260810 06:46:57.193921 16291 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 06:46:57.759868 16291 mini_ranger.cc:87] Created ranger Postgres database
I20260810 06:46:57.759967 16291 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-admin
2026-08-10 06:46:58,054 [I] DB FLAVOR :POSTGRES
2026-08-10 06:46:58,054 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:46:58,055 [I] Checking connection..
2026-08-10 06:46:58,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 06:46:58,349 [I] Checking connection passed.
2026-08-10 06:46:58,350 [I] --------- Verifying version history table ---------
2026-08-10 06:46:58,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:46:58,619 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:46:58,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:46:58,882 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:46:58,882 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 06:46:58,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 06:46:59.166 UTC [29769] WARNING: there is no transaction in progress
2026-08-10 06:46:59,174 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 06:46:59,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:46:59,448 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 06:46:59,448 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:46:59,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 06:46:59,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 06:47:00,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vxj2.c.gcp-upstream.internal','N') ;"
2026-08-10 06:47:00,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:47:00,531 [I] Table x_portal_user does not exist in database ranger
2026-08-10 06:47:00,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:47:00,810 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 06:47:00,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:47:01,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:47:01,340 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 06:47:01,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 06:47:02.462 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.480 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.501 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.518 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.537 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.556 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.618 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.627 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.644 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.656 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.667 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.685 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.696 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.703 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.712 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:02.719 UTC [29899] WARNING: there is no transaction in progress
2026-08-10 06:47:03,528 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 06:47:03,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 06:47:03,778 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 06:47:03,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:47:04,085 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 06:47:04,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:47:04,352 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 06:47:04,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:47:04,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:47:04,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vxj2.c.gcp-upstream.internal';"
2026-08-10 06:47:05,141 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 06:47:05,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 06:47:05,438 [I] DB_PATCHES have already been applied
I20260810 06:47:05.445358 16291 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktpUSKt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 06:47:05.445432 16291 mini_ranger.cc:193] Using host: 127.15.232.212
I20260810 06:47:05.449093 16291 mini_ranger.cc:227] Ranger admin URL: http://127.15.232.212:42801
Aug 10, 2026 6:47:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:47:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:47:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:47:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:47:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:47:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:47:06 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 06:47:06.743456 16291 mini_ranger.cc:160] Time spent starting Ranger: real 9.568s user 0.000s sys 0.010s
Aug 10, 2026 6:47:27 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:47:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 6:47:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:47:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:47:27 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:47:28 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 06:47:29.309914 16291 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.232.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.232.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/rangerkms_127.15.232.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.232.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/rangerkms_127.15.232.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.232.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.232.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.232.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/HTTP_127.15.232.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.232.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/HTTP_127.15.232.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 06:47:29 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344449, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 06:47:29.404991 16291 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 06:47:29.416146 16291 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 06:47:30.392244 16291 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 06:47:30.392340 16291 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-kms
I20260810 06:47:30.392364 16291 mini_ranger_kms.cc:227] Using postgres at 127.15.232.212:42523
2026-08-10 06:47:30,585 [I] DB FLAVOR :POSTGRES
2026-08-10 06:47:30,586 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:47:30,586 [I] Checking connection
2026-08-10 06:47:30,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 06:47:30,831 [I] connection success
2026-08-10 06:47:30,831 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 06:47:30,832 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 06:47:30,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 06:47:31,117 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 06:47:31,118 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:47:31,118 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 06:47:31,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.212:42523/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 06:47:31,394 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 06:47:31.925454 16291 mini_ranger_kms.cc:310] Created kms service
I20260810 06:47:32.147861 16291 mini_ranger_kms.cc:326] Created kudu user
I20260810 06:47:32.201637 16291 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 06:47:32 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344449, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:47:32.672418 16291 mini_ranger_kms.cc:384] Added ranger policy
I20260810 06:47:32.672657 16291 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktpUSKt/build/release/bin/postgresql.jar:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktpUSKt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 06:47:32.672681 16291 mini_ranger_kms.cc:240] Using host: 127.15.232.212
I20260810 06:47:32.676406 16291 mini_ranger_kms.cc:276] Ranger KMS PID: 30107
I20260810 06:47:32.676501 16291 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.232.212:49505
Aug 10, 2026 6:47:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:47:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:47:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:47:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:47:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:47:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260810 06:47:35.174088 16291 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.769s user 0.000s sys 0.173s
I20260810 06:47:35.174196 16291 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 06:47:35.174245 16291 mini_ranger_kms.cc:401] 127.15.232.212:49505/kms/v1/keys
Aug 10 06:47:35 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344455, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.232.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 06:47:35 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344455, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.232.212@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
Aug 10 06:47:36 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344455, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.232.212@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
Aug 10 06:47:38 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:47:38.749389 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:44293
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:44293
--ranger_config_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.212:49505/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:38.833635 30166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:38.833894 30166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:38.833932 30166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:38.835109 30166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:47:38.835186 30166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:38.835213 30166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:38.835245 30166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:47:38.835270 30166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:47:38.836879 30166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.232.212:49505/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:44293
--ranger_config_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:44293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:38.837215 30166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:38.837452 30166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:38.842685 30171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:38.842676 30174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:38.842723 30172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:38.842916 30166 server_base.cc:1034] running on GCE node
I20260810 06:47:38.843282 30166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:38.843547 30166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:38.844686 30166 hybrid_clock.cc:648] HybridClock initialized: now 1786344458844661 us; error 45 us; skew 500 ppm
Aug 10 06:47:38 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344458, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:38.847272 30166 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:47:38.847729 30166 webserver.cc:460] Webserver started at http://127.15.232.254:39007/ using document root <none> and password file <none>
I20260810 06:47:38.847966 30166 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:38.848008 30166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:38.848156 30166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:47:38 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344458, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:47:38.899605 30166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "68b3c9f809e14e6da239160c42ef4346"
format_stamp: "Formatted at 2026-08-10 06:47:38 on dist-test-slave-vxj2"
server_key: "3094fb2f3d70e3654425c879d4555e25"
server_key_iv: "08bfd681e6bc606dbcff45e6729a6e56"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:38.900107 30166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "68b3c9f809e14e6da239160c42ef4346"
format_stamp: "Formatted at 2026-08-10 06:47:38 on dist-test-slave-vxj2"
server_key: "3094fb2f3d70e3654425c879d4555e25"
server_key_iv: "08bfd681e6bc606dbcff45e6729a6e56"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:38.902091 30166 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:47:38.915175 30182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:38.915459 30166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:38.915575 30166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "68b3c9f809e14e6da239160c42ef4346"
format_stamp: "Formatted at 2026-08-10 06:47:38 on dist-test-slave-vxj2"
server_key: "3094fb2f3d70e3654425c879d4555e25"
server_key_iv: "08bfd681e6bc606dbcff45e6729a6e56"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:38.915699 30166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:38.930182 30166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:38.932508 30166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:38.932664 30166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:38.938022 30166 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:44293
I20260810 06:47:38.938057 30240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:44293 every 8 connection(s)
I20260810 06:47:38.938421 30166 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:47:38.939260 30241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:38.942482 30241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346: Bootstrap starting.
I20260810 06:47:38.943221 30241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:38.943547 30241 log.cc:824] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:38.944355 30241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346: No bootstrap required, opened a new log
I20260810 06:47:38.945592 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 30166
I20260810 06:47:38.945710 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:47:38.945890 16291 external_mini_cluster.cc:1428] Setting key 1abed105175ac94f6e0fe253fe7f740f
I20260810 06:47:38.947219 30241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b3c9f809e14e6da239160c42ef4346" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44293 } }
I20260810 06:47:38.947433 30241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:38.947501 30241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b3c9f809e14e6da239160c42ef4346, State: Initialized, Role: FOLLOWER
I20260810 06:47:38.947654 30241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b3c9f809e14e6da239160c42ef4346" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44293 } }
I20260810 06:47:38.947736 30241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:47:38.947765 30241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:47:38.947839 30241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:38.948514 30241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b3c9f809e14e6da239160c42ef4346" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44293 } }
I20260810 06:47:38.948623 30241 leader_election.cc:302] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68b3c9f809e14e6da239160c42ef4346; no voters:
I20260810 06:47:38.948834 30241 leader_election.cc:288] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:47:38.948882 30247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:47:38.949118 30247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 1 LEADER]: Becoming Leader. State: Replica: 68b3c9f809e14e6da239160c42ef4346, State: Running, Role: LEADER
I20260810 06:47:38.949193 30241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:47:38.949298 30247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b3c9f809e14e6da239160c42ef4346" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44293 } }
I20260810 06:47:38.949652 30247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68b3c9f809e14e6da239160c42ef4346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b3c9f809e14e6da239160c42ef4346" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44293 } } }
I20260810 06:47:38.949843 30249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68b3c9f809e14e6da239160c42ef4346. Latest consensus state: current_term: 1 leader_uuid: "68b3c9f809e14e6da239160c42ef4346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b3c9f809e14e6da239160c42ef4346" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44293 } } }
I20260810 06:47:38.950716 30249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:38.950793 30247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:38.950708 30241 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:47:38.951444 30261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:47:38.951956 30261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:47:38 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:38.960872 30246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:38.946436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.15.232.254:44293)
0810 06:47:38.946621 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:38.946631 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:38.946680 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:47:38.947195 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:38.947202 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:38.947268 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:38.947419 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:38.948317 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:38.949713 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:38.950675 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:38.951330 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:38.951438 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:38.954860 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:38.954917 (+ 57us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:38.954918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:38.955019 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:38.958301 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:38.958815 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:38.958819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:38.958819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:38.958953 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:38.959177 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:38.959178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:38.959178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:38.959531 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:38.959690 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:38.960171 (+ 481us) server_negotiation.cc:299] Negotiation successful
0810 06:47:38.960352 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
I20260810 06:47:38.964653 30261 catalog_manager.cc:1347] Generated new cluster ID: af02095909ed4c3aa5098f87d18bec55
I20260810 06:47:38.964720 30261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:47:38.993919 30261 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:47:38.994804 30261 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:47:39.007323 30261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346: Generated new TSK 0
I20260810 06:47:39.007563 30261 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:47:39.738597 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.212:49505/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44293
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:39.830988 30291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:39.831243 30291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:39.831285 30291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:39.832470 30291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:39.832544 30291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:39.832610 30291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:47:39.834210 30291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.232.212:49505/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:39.834489 30291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:39.834738 30291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:39.840513 30296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:39.840539 30300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:39.840922 30298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:39.842036 30291 server_base.cc:1034] running on GCE node
I20260810 06:47:39.842290 30291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:39.842572 30291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:39.843796 30291 hybrid_clock.cc:648] HybridClock initialized: now 1786344459843763 us; error 45 us; skew 500 ppm
Aug 10 06:47:39 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344459, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:39.847101 30291 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:47:39.847630 30291 webserver.cc:460] Webserver started at http://127.15.232.193:37911/ using document root <none> and password file <none>
I20260810 06:47:39.847869 30291 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:39.847939 30291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:39.848096 30291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:47:39 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344459, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:47:39.865847 30291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "9d626d15b7104c38918e55312eab75f1"
format_stamp: "Formatted at 2026-08-10 06:47:39 on dist-test-slave-vxj2"
server_key: "b8721f39e0bec2f4e639700a15333d45"
server_key_iv: "0330dcdc88c0dd86934bca183c9adb1e"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:39.866333 30291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "9d626d15b7104c38918e55312eab75f1"
format_stamp: "Formatted at 2026-08-10 06:47:39 on dist-test-slave-vxj2"
server_key: "b8721f39e0bec2f4e639700a15333d45"
server_key_iv: "0330dcdc88c0dd86934bca183c9adb1e"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:39.868273 30291 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:47:39.884835 30307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:39.885174 30291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:39.885294 30291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "9d626d15b7104c38918e55312eab75f1"
format_stamp: "Formatted at 2026-08-10 06:47:39 on dist-test-slave-vxj2"
server_key: "b8721f39e0bec2f4e639700a15333d45"
server_key_iv: "0330dcdc88c0dd86934bca183c9adb1e"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:39.885398 30291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:39.905893 30291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:39.908207 30291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:39.908387 30291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:39.908696 30291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:39.909147 30291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:39.909233 30291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:39.909276 30291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:39.909303 30291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:39.916617 30291 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:33229
I20260810 06:47:39.917064 30291 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:47:39.917182 30432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:33229 every 8 connection(s)
Aug 10 06:47:39 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344459, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:39.926705 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 30291
I20260810 06:47:39.926802 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:47:39.927002 16291 external_mini_cluster.cc:1428] Setting key 92583513ca94e8decc135a203f19176f
I20260810 06:47:39.929288 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:39.917956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44293 (local address 127.15.232.193:40253)
0810 06:47:39.918373 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:39.918423 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:39.919458 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:39.919925 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:39.919929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:39.920174 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:39.920434 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:39.920439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:39.921372 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:39.921373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:39.922619 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:39.922622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:39.923063 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:39.923064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:39.923201 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:39.923878 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:39.923889 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:39.925411 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:39.927511 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:39.927521 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:39.927524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:39.927869 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:39.928092 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:39.928093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:39.928094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:39.928240 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:39.928551 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:39.928557 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:39.928721 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 06:47:39.928848 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 06:47:39.929008 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":114,"threads_started":1}
I20260810 06:47:39.929893 30438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:39.917991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:40253 (local address 127.15.232.254:44293)
0810 06:47:39.919665 (+ 1674us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:39.919668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:39.919677 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:47:39.919706 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:39.919707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:39.919721 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:39.919801 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:39.920585 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:39.921245 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:39.922745 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:39.922959 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:39.923043 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:39.925517 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:39.925519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:39.925519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:39.925527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:39.927411 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:39.927948 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:39.927949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:39.927949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:39.928012 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:39.928318 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:39.928319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:39.928319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:39.928458 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:39.928522 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:39.929661 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0810 06:47:39.929789 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1641,"thread_start_us":103,"threads_started":1}
I20260810 06:47:39.930503 30433 heartbeater.cc:347] Connected to a master server at 127.15.232.254:44293
I20260810 06:47:39.930629 30433 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:39.930823 30433 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
I20260810 06:47:39.931397 30201 ts_manager.cc:194] Registered new tserver with Master: 9d626d15b7104c38918e55312eab75f1 (127.15.232.193:33229)
I20260810 06:47:39.932017 30201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:40253
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:47:39.961225 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.212:49505/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44293
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:40.078680 30443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:40.078989 30443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:40.079027 30443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:40.080879 30443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:40.080955 30443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:40.080998 30443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:47:40.083405 30443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.232.212:49505/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:40.083685 30443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:40.084031 30443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:40.091202 30452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:40.091411 30451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:40.091171 30459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:40.092690 30443 server_base.cc:1034] running on GCE node
I20260810 06:47:40.092964 30443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:40.093364 30443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:40.094554 30443 hybrid_clock.cc:648] HybridClock initialized: now 1786344460094511 us; error 58 us; skew 500 ppm
Aug 10 06:47:40 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344460, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:40.098300 30443 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:47:40.098948 30443 webserver.cc:460] Webserver started at http://127.15.232.194:45067/ using document root <none> and password file <none>
I20260810 06:47:40.099227 30443 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:40.099303 30443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:40.099442 30443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:47:40 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344460, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:47:40.116021 30443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "cdb889b2d02946b0836af86d1a484833"
format_stamp: "Formatted at 2026-08-10 06:47:40 on dist-test-slave-vxj2"
server_key: "9ee71e418a9b19d8a73824af87d68edc"
server_key_iv: "cb0cbdf2ac64e8878f3c997fa601e257"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:40.116557 30443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "cdb889b2d02946b0836af86d1a484833"
format_stamp: "Formatted at 2026-08-10 06:47:40 on dist-test-slave-vxj2"
server_key: "9ee71e418a9b19d8a73824af87d68edc"
server_key_iv: "cb0cbdf2ac64e8878f3c997fa601e257"
server_key_version: "kuduclusterkey@0"
Aug 10 06:47:40 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344460, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:40.129647 30443 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.000s sys 0.004s
I20260810 06:47:40.147049 30467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:40.147315 30443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:47:40.147429 30443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "cdb889b2d02946b0836af86d1a484833"
format_stamp: "Formatted at 2026-08-10 06:47:40 on dist-test-slave-vxj2"
server_key: "9ee71e418a9b19d8a73824af87d68edc"
server_key_iv: "cb0cbdf2ac64e8878f3c997fa601e257"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:40.147534 30443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:40.163712 30443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:40.166491 30443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:40.166726 30443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:40.167078 30443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:40.167594 30443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:40.167697 30443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:40.167774 30443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:40.167833 30443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:40.175859 30443 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:45071
I20260810 06:47:40.176288 30443 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:47:40.176960 30603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:45071 every 8 connection(s)
Aug 10 06:47:40 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344460, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:40.185331 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 30443
I20260810 06:47:40.185446 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:47:40.185633 16291 external_mini_cluster.cc:1428] Setting key b4cd346ba0b133f28d120e85adfca4f6
I20260810 06:47:40.189993 30438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:40.177598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:41597 (local address 127.15.232.254:44293)
0810 06:47:40.177652 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:40.177655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:40.177667 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:47:40.178244 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:40.178245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:40.178268 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:40.178317 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:40.178947 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:40.179555 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:40.180597 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:40.180836 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:40.180929 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:40.183232 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:40.183234 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:40.183234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:40.183243 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:40.184937 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:40.188705 (+ 3768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:40.188707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:40.188707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:40.188807 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:40.189145 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:40.189164 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:40.189164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:40.189331 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:40.189396 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:40.189768 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 06:47:40.189882 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:47:40.191653 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:40.177238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44293 (local address 127.15.232.194:41597)
0810 06:47:40.177439 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:40.177480 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:40.178140 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:40.178401 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:40.178407 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:40.178626 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:40.178842 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:40.178846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:40.179665 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:40.179666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:40.180496 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:40.180498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:40.180957 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:40.180958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:40.181091 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:40.181751 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:40.181765 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:40.183118 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:40.185041 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:40.185050 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:40.185052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:40.188580 (+ 3528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:40.188893 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:40.188894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:40.188894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:40.189060 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:40.189442 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:40.189448 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:40.189587 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 06:47:40.189706 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 06:47:40.189892 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":105,"threads_started":1}
I20260810 06:47:40.193526 30604 heartbeater.cc:347] Connected to a master server at 127.15.232.254:44293
I20260810 06:47:40.193634 30604 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:40.193852 30604 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
I20260810 06:47:40.194343 30201 ts_manager.cc:194] Registered new tserver with Master: cdb889b2d02946b0836af86d1a484833 (127.15.232.194:45071)
I20260810 06:47:40.194824 30201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:41597
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:47:40.220471 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.212:49505/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44293
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:40.340621 30611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:40.341019 30611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:40.341126 30611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:40.343031 30611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:40.343153 30611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:40.343251 30611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:47:40.345590 30611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.232.212:49505/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:40.346120 30611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:40.346527 30611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:40.353721 30617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:40.354137 30616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:40.354396 30619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:40.356158 30611 server_base.cc:1034] running on GCE node
I20260810 06:47:40.356520 30611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:40.356920 30611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:40.358112 30611 hybrid_clock.cc:648] HybridClock initialized: now 1786344460358083 us; error 42 us; skew 500 ppm
Aug 10 06:47:40 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344460, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:40.362028 30611 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:47:40.362697 30611 webserver.cc:460] Webserver started at http://127.15.232.195:33051/ using document root <none> and password file <none>
I20260810 06:47:40.363057 30611 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:40.363200 30611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:40.363457 30611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:47:40 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344460, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:47:40.379971 30611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "f347da363343445787f4134c94cd212f"
format_stamp: "Formatted at 2026-08-10 06:47:40 on dist-test-slave-vxj2"
server_key: "b5fdc2ed17b1741e12c2befb5fab8cbd"
server_key_iv: "1b86bef6bac88d386e6a4e4f0c0f7a76"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:40.380623 30611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "f347da363343445787f4134c94cd212f"
format_stamp: "Formatted at 2026-08-10 06:47:40 on dist-test-slave-vxj2"
server_key: "b5fdc2ed17b1741e12c2befb5fab8cbd"
server_key_iv: "1b86bef6bac88d386e6a4e4f0c0f7a76"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:40.389590 30611 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.002s
I20260810 06:47:40.412549 30626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:40.412833 30611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:40.412930 30611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "f347da363343445787f4134c94cd212f"
format_stamp: "Formatted at 2026-08-10 06:47:40 on dist-test-slave-vxj2"
server_key: "b5fdc2ed17b1741e12c2befb5fab8cbd"
server_key_iv: "1b86bef6bac88d386e6a4e4f0c0f7a76"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:40.413024 30611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:40.437722 30611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:40.440266 30611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:40.440425 30611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:40.440658 30611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:40.441079 30611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:40.441129 30611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:40.441185 30611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:40.441205 30611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:40.448803 30611 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:45031
I20260810 06:47:40.449261 30757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:45031 every 8 connection(s)
I20260810 06:47:40.449419 30611 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:47:40.456951 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 30611
I20260810 06:47:40.457087 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:47:40.457333 16291 external_mini_cluster.cc:1428] Setting key 9fd7e8c73d9b5e3438e894d17581a697
Aug 10 06:47:40 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344460, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:40.462949 30438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:40.450922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:47963 (local address 127.15.232.254:44293)
0810 06:47:40.451013 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:40.451015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:40.452396 (+ 1381us) server_negotiation.cc:407] Connection header received
0810 06:47:40.453119 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:40.453121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:40.453136 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:40.453308 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:40.454132 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:40.454879 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:40.456316 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:40.456626 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:40.456705 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:40.459354 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:40.459356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:40.459356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:40.459365 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:40.461132 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:40.461688 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:40.461689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:40.461690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:40.461768 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:40.462084 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:40.462085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:40.462085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:40.462237 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:40.462310 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:40.462688 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 06:47:40.462813 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:47:40.463071 30760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:40.450591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44293 (local address 127.15.232.195:47963)
0810 06:47:40.452210 (+ 1619us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:40.452271 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:40.452983 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:40.453435 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:40.453441 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:40.453725 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:40.454013 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:40.454018 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:40.454997 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:40.454999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:40.456212 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:40.456214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:40.456749 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:40.456750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:40.456911 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:40.457610 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:40.457621 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:40.459249 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:40.461245 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:40.461254 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:40.461257 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:40.461598 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:40.461851 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:40.461852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:40.461853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:40.462007 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:40.462324 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:40.462331 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:40.462517 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 06:47:40.462643 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 06:47:40.462817 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1561,"thread_start_us":96,"threads_started":1}
I20260810 06:47:40.464390 30758 heartbeater.cc:347] Connected to a master server at 127.15.232.254:44293
I20260810 06:47:40.464509 30758 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:40.464721 30758 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
I20260810 06:47:40.465245 30201 ts_manager.cc:194] Registered new tserver with Master: f347da363343445787f4134c94cd212f (127.15.232.195:45031)
I20260810 06:47:40.465927 30201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:47963
I20260810 06:47:40.469256 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:47:40.470109 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 30291
I20260810 06:47:40.486186 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 30443
I20260810 06:47:40.495074 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 30611
I20260810 06:47:40.509891 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 30166
I20260810 06:47:40.518683 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:44293
--webserver_interface=127.15.232.254
--webserver_port=39007
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:44293
--ranger_config_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.212:49505/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.232.212:49505/kms with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:40.605127 30761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:40.605460 30761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:40.605504 30761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:40.606774 30761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:47:40.606853 30761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:40.606876 30761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:40.606899 30761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:47:40.606925 30761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:47:40.608649 30761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.232.212:49505/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:44293
--ranger_config_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:44293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=39007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:40.608955 30761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:40.609251 30761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:40.615388 30769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:40.615408 30767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:40.615564 30761 server_base.cc:1034] running on GCE node
W20260810 06:47:40.615450 30766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:40.616258 30761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:40.616593 30761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:40.617826 30761 hybrid_clock.cc:648] HybridClock initialized: now 1786344460617758 us; error 34 us; skew 500 ppm
Aug 10 06:47:40 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344460, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:40.620951 30761 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:47:40.621578 30761 webserver.cc:460] Webserver started at http://127.15.232.254:39007/ using document root <none> and password file <none>
I20260810 06:47:40.621848 30761 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:40.621935 30761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:47:41 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344460, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:47:41.475013 30761 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:41.476058 30777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:41.476372 30761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:41.476503 30761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "68b3c9f809e14e6da239160c42ef4346"
format_stamp: "Formatted at 2026-08-10 06:47:38 on dist-test-slave-vxj2"
server_key: "3094fb2f3d70e3654425c879d4555e25"
server_key_iv: "08bfd681e6bc606dbcff45e6729a6e56"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:41.477008 30761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:41.490851 30761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:41.492740 30761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:41.492924 30761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:41.497767 30761 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:44293
I20260810 06:47:41.497795 30842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:44293 every 8 connection(s)
I20260810 06:47:41.498209 30761 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:47:41.499505 30843 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:47:41.500201 30843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346: Bootstrap starting.
I20260810 06:47:41.500671 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 30761
I20260810 06:47:41.501005 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.212:49505/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:33229
--local_ip_for_outbound_sockets=127.15.232.193
--tserver_master_addrs=127.15.232.254:44293
--webserver_port=37911
--webserver_interface=127.15.232.193
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.232.212:49505/kms with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:47:41.508646 30843 log.cc:824] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:41.510509 30843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:47:41.510828 30843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346: Bootstrap complete.
I20260810 06:47:41.513739 30843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b3c9f809e14e6da239160c42ef4346" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44293 } }
I20260810 06:47:41.514040 30843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b3c9f809e14e6da239160c42ef4346, State: Initialized, Role: FOLLOWER
I20260810 06:47:41.514319 30843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b3c9f809e14e6da239160c42ef4346" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44293 } }
I20260810 06:47:41.514415 30843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:47:41.514463 30843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:47:41.514525 30843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:47:41.515661 30843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b3c9f809e14e6da239160c42ef4346" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44293 } }
I20260810 06:47:41.515805 30843 leader_election.cc:302] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68b3c9f809e14e6da239160c42ef4346; no voters:
I20260810 06:47:41.516044 30843 leader_election.cc:288] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:47:41.516088 30848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:47:41.516227 30848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [term 2 LEADER]: Becoming Leader. State: Replica: 68b3c9f809e14e6da239160c42ef4346, State: Running, Role: LEADER
I20260810 06:47:41.516357 30843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:47:41.516343 30848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b3c9f809e14e6da239160c42ef4346" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44293 } }
I20260810 06:47:41.516701 30848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68b3c9f809e14e6da239160c42ef4346. Latest consensus state: current_term: 2 leader_uuid: "68b3c9f809e14e6da239160c42ef4346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b3c9f809e14e6da239160c42ef4346" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44293 } } }
I20260810 06:47:41.516785 30848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:41.516935 30849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "68b3c9f809e14e6da239160c42ef4346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b3c9f809e14e6da239160c42ef4346" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44293 } } }
I20260810 06:47:41.517032 30849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:41.518576 30843 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:47:41.518606 30853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:47:41.522281 30853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:47:41.522634 30853 catalog_manager.cc:1259] Loaded cluster ID: af02095909ed4c3aa5098f87d18bec55
I20260810 06:47:41.522710 30853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:47:41.524269 30853 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:47:41.524578 30853 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346: Loaded TSK: 0
I20260810 06:47:41.524871 30853 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:47:41.617457 30846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:41.617766 30846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:41.617811 30846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:41.619614 30846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:41.619686 30846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:41.619728 30846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:47:41.622076 30846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.232.212:49505/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:33229
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=37911
--tserver_master_addrs=127.15.232.254:44293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:41.622344 30846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:41.622615 30846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:41.629935 30891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:41.629885 30889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:41.630250 30887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:41.630995 30846 server_base.cc:1034] running on GCE node
I20260810 06:47:41.631233 30846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:41.631539 30846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:41.632732 30846 hybrid_clock.cc:648] HybridClock initialized: now 1786344461632709 us; error 36 us; skew 500 ppm
Aug 10 06:47:41 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344461, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:41.639791 30846 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:47:41.640381 30846 webserver.cc:460] Webserver started at http://127.15.232.193:37911/ using document root <none> and password file <none>
I20260810 06:47:41.640684 30846 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:41.640772 30846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:47:41 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344461, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:47:41.669248 30846 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:47:41.670535 30901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:41.670799 30846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:47:41.670912 30846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "9d626d15b7104c38918e55312eab75f1"
format_stamp: "Formatted at 2026-08-10 06:47:39 on dist-test-slave-vxj2"
server_key: "b8721f39e0bec2f4e639700a15333d45"
server_key_iv: "0330dcdc88c0dd86934bca183c9adb1e"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:41.671415 30846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:41.700814 30846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:41.703548 30846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:41.703742 30846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:41.704047 30846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:41.704463 30846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:41.704520 30846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:41.704563 30846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:41.704589 30846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:41.711956 30846 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:33229
I20260810 06:47:41.712484 30846 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:47:41.715389 31028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:33229 every 8 connection(s)
I20260810 06:47:41.718530 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 30846
I20260810 06:47:41.718849 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.212:49505/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:45071
--local_ip_for_outbound_sockets=127.15.232.194
--tserver_master_addrs=127.15.232.254:44293
--webserver_port=45067
--webserver_interface=127.15.232.194
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.232.212:49505/kms with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:47:41 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344461, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:41.731025 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:41.716230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44293 (local address 127.15.232.193:40949)
0810 06:47:41.717294 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:41.717348 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:41.718177 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:41.718932 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:41.718936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:41.719205 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:41.719499 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:41.719507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:41.721213 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:41.721214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:41.722500 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:41.722504 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:41.723490 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:41.723491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:41.723633 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:41.724301 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:41.724312 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:41.726170 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:41.728784 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:41.728794 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:41.728797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:41.729117 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:41.729486 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:41.729486 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:41.729487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:41.729626 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:41.730278 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:41.730287 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:41.730469 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 06:47:41.730588 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 06:47:41.730762 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"thread_start_us":97,"threads_started":1}
I20260810 06:47:41.731549 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:41.716309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:40949 (local address 127.15.232.254:44293)
0810 06:47:41.716554 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:41.716565 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:41.717511 (+ 946us) server_negotiation.cc:407] Connection header received
0810 06:47:41.718459 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:41.718468 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:41.718561 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:41.718709 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:41.719644 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:41.721038 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:41.722610 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:41.723229 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:41.723319 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:41.726324 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:41.726330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:41.726330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:41.726345 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:41.728675 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:41.729265 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:41.729269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:41.729270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:41.729393 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:41.729761 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:41.729762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:41.729763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:41.730172 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:41.730283 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:41.730710 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 06:47:41.730877 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":92,"threads_started":1}
I20260810 06:47:41.732686 31029 heartbeater.cc:347] Connected to a master server at 127.15.232.254:44293
I20260810 06:47:41.732802 31029 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:41.733007 31029 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
W20260810 06:47:41.733924 31029 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:44293 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:47:41.734046 31029 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:41.734124 31029 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
I20260810 06:47:41.734474 31029 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:41.734561 31029 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
W20260810 06:47:41.734905 31029 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:47:41.847210 31035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:41.847528 31035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:41.847573 31035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:41.849401 31035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:41.849473 31035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:41.849519 31035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:47:41.851784 31035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.232.212:49505/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:45071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=45067
--tserver_master_addrs=127.15.232.254:44293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:41.852072 31035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:41.852401 31035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:41.858270 31044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:41.858282 31042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:41.858698 31041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:41.859510 31035 server_base.cc:1034] running on GCE node
I20260810 06:47:41.859745 31035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:41.860054 31035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:41.861267 31035 hybrid_clock.cc:648] HybridClock initialized: now 1786344461861236 us; error 47 us; skew 500 ppm
Aug 10 06:47:41 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344461, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:41.864079 31035 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:47:41.864614 31035 webserver.cc:460] Webserver started at http://127.15.232.194:45067/ using document root <none> and password file <none>
I20260810 06:47:41.864858 31035 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:41.864919 31035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:47:41 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344461, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:47:41.879768 31035 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:41.881743 31052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:41.881928 31035 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:47:41.882025 31035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "cdb889b2d02946b0836af86d1a484833"
format_stamp: "Formatted at 2026-08-10 06:47:40 on dist-test-slave-vxj2"
server_key: "9ee71e418a9b19d8a73824af87d68edc"
server_key_iv: "cb0cbdf2ac64e8878f3c997fa601e257"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:41.882552 31035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:41.905586 31035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:41.907930 31035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:41.908115 31035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:41.908381 31035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:41.908808 31035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:41.908878 31035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:41.908922 31035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:41.908944 31035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:41.916589 31035 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:45071
I20260810 06:47:41.916760 31180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:45071 every 8 connection(s)
I20260810 06:47:41.917119 31035 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:47:41.917589 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 31035
I20260810 06:47:41.917933 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.212:49505/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:45031
--local_ip_for_outbound_sockets=127.15.232.195
--tserver_master_addrs=127.15.232.254:44293
--webserver_port=33051
--webserver_interface=127.15.232.195
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.232.212:49505/kms with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:47:41 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344461, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:41.928721 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:41.918188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:50433 (local address 127.15.232.254:44293)
0810 06:47:41.918240 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:41.918243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:41.918569 (+ 326us) server_negotiation.cc:407] Connection header received
0810 06:47:41.919934 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:41.919936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:41.919956 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:41.919989 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:41.920944 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:41.921536 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:41.922629 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:41.922891 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:41.922978 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:41.925310 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:41.925312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:41.925312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:41.925321 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:41.927000 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:41.927454 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:41.927455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:41.927456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:41.927518 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:41.927798 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:41.927799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:41.927799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:41.927971 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:41.928038 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:41.928535 (+ 497us) server_negotiation.cc:299] Negotiation successful
0810 06:47:41.928623 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:47:41.929144 31182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:41.918190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44293 (local address 127.15.232.194:50433)
0810 06:47:41.918371 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:41.918436 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:41.919184 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:41.920204 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:41.920208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:41.920481 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:41.920775 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:41.920782 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:41.921645 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:41.921646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:41.922512 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:41.922515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:41.923012 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:41.923014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:41.923144 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:41.923763 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:41.923772 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:41.925158 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:41.927093 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:41.927101 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:41.927104 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:41.927363 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:41.927592 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:41.927593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:41.927593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:41.927722 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:41.928069 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:41.928076 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:41.928318 (+ 242us) client_negotiation.cc:769] Sending connection context
0810 06:47:41.928448 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 06:47:41.928626 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260810 06:47:41.929895 31181 heartbeater.cc:347] Connected to a master server at 127.15.232.254:44293
I20260810 06:47:41.930009 31181 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:41.930214 31181 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
W20260810 06:47:41.930950 31181 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:44293 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:47:41.931025 31181 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:41.931108 31181 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
I20260810 06:47:41.931495 31181 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:41.931576 31181 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
W20260810 06:47:41.931921 31181 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:47:42.047174 31183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:42.047500 31183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:42.047542 31183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:42.049563 31183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:42.049643 31183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:42.049690 31183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:47:42.052165 31183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.232.212:49505/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:45031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=33051
--tserver_master_addrs=127.15.232.254:44293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:42.052482 31183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:42.052783 31183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:42.060724 31191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:42.060825 31192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:42.060731 31194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:42.061285 31183 server_base.cc:1034] running on GCE node
I20260810 06:47:42.061546 31183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:42.061843 31183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:42.063048 31183 hybrid_clock.cc:648] HybridClock initialized: now 1786344462063020 us; error 36 us; skew 500 ppm
Aug 10 06:47:42 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344462, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:42.066431 31183 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:47:42.067045 31183 webserver.cc:460] Webserver started at http://127.15.232.195:33051/ using document root <none> and password file <none>
I20260810 06:47:42.067303 31183 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:42.067441 31183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:47:42 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344462, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:47:42.086572 31183 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:42.087826 31203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:42.088086 31183 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:42.088205 31183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "f347da363343445787f4134c94cd212f"
format_stamp: "Formatted at 2026-08-10 06:47:40 on dist-test-slave-vxj2"
server_key: "b5fdc2ed17b1741e12c2befb5fab8cbd"
server_key_iv: "1b86bef6bac88d386e6a4e4f0c0f7a76"
server_key_version: "kuduclusterkey@0"
I20260810 06:47:42.088706 31183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:42.119668 31183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:42.122021 31183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:42.122206 31183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:42.122516 31183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:42.122905 31183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:42.122949 31183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:42.122972 31183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:42.122985 31183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:42.131101 31183 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:45031
I20260810 06:47:42.131240 31340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:45031 every 8 connection(s)
I20260810 06:47:42.131647 31183 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:47:42.139300 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 31183
Aug 10 06:47:42 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344462, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:42.144556 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:42.132655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44293 (local address 127.15.232.195:46845)
0810 06:47:42.132946 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:42.132983 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:42.133518 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:42.133830 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:42.133833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:42.134094 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:42.134327 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:42.134332 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.135119 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.135120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:42.136039 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:42.136042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.136623 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.136625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:42.136770 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:42.137851 (+ 1081us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:42.137862 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:42.140560 (+ 2698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:42.142497 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:42.142506 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:42.142509 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:42.142893 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:42.143112 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:42.143113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:42.143113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:42.143274 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:42.143738 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:42.143748 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:42.143959 (+ 211us) client_negotiation.cc:769] Sending connection context
0810 06:47:42.144093 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 06:47:42.144284 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":97,"threads_started":1}
I20260810 06:47:42.144938 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:42.132676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:46845 (local address 127.15.232.254:44293)
0810 06:47:42.132739 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:42.132741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:42.133090 (+ 349us) server_negotiation.cc:407] Connection header received
0810 06:47:42.133644 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:42.133647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:42.133660 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:42.133753 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:42.134442 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.135003 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.136129 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.136439 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.136511 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:42.140674 (+ 4163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:42.140676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:42.140677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:42.140686 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:42.142398 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:42.142968 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:42.142970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:42.142970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:42.143036 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:42.143377 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:42.143378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:42.143379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:42.143613 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:42.143701 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:42.144739 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0810 06:47:42.144838 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 06:47:42.145326 31341 heartbeater.cc:347] Connected to a master server at 127.15.232.254:44293
I20260810 06:47:42.145442 31341 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:42.145651 31341 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
W20260810 06:47:42.146742 31341 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:44293 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:47:42.146824 31341 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:42.146888 31341 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
I20260810 06:47:42.147292 31341 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:42.147526 31341 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
W20260810 06:47:42.147977 31341 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:47:42.155162 31344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:42.139796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.15.232.254:44293)
0810 06:47:42.140177 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:42.140179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:42.140187 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:47:42.140211 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:42.140212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:42.140226 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:42.141059 (+ 833us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:42.143436 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.144060 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.149534 (+ 5474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.149874 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.149943 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:42.151923 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:42.151925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:42.151926 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:42.151944 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:42.153806 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:42.154189 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:42.154191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:42.154191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:42.154271 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:42.154508 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:42.154509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:42.154509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:42.154668 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:42.154875 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:42.154927 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 06:47:42.155044 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":104,"threads_started":1}
Aug 10 06:47:42 dist-test-slave-vxj2 krb5kdc[29649](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344462, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:42.735136 31029 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:42.735308 31029 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
I20260810 06:47:42.735791 30798 ts_manager.cc:194] Registered new tserver with Master: 9d626d15b7104c38918e55312eab75f1 (127.15.232.193:33229)
I20260810 06:47:42.736800 30798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:40949
I20260810 06:47:42.932091 31181 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:42.932235 31181 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
I20260810 06:47:42.932579 30798 ts_manager.cc:194] Registered new tserver with Master: cdb889b2d02946b0836af86d1a484833 (127.15.232.194:45071)
I20260810 06:47:42.933001 30798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:50433
Aug 10 06:47:43 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344462, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:47:43.148193 31341 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:43.148391 31341 heartbeater.cc:511] Master 127.15.232.254:44293 requested a full tablet report, sending...
I20260810 06:47:43.148772 30798 ts_manager.cc:194] Registered new tserver with Master: f347da363343445787f4134c94cd212f (127.15.232.195:45031)
I20260810 06:47:43.149338 30798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:46845
I20260810 06:47:43.154855 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:47:43.162778 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.155916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46210 (local address 127.15.232.254:44293)
0810 06:47:43.156176 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.156178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.156183 (+ 5us) server_negotiation.cc:407] Connection header received
0810 06:47:43.156202 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.156203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.156212 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.156293 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:43.156880 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.157466 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.158186 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.158389 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.158483 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.159815 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:43.159817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:43.159817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:43.159825 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:43.161542 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.161942 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.161943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.161944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.162008 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.162223 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.162223 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.162224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.162345 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:43.162464 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.162567 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.162656 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":90,"threads_started":1}
I20260810 06:47:43.164994 30798 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:47:43.166851 30798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:47:43.178553 31377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.173274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:33229 (local address 127.0.0.1:49344)
0810 06:47:43.173757 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.173808 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:43.173883 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:43.174237 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:43.174241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:43.174244 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:43.174343 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.174345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.175715 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.175716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.177223 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.177226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.178241 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.178242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.178331 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.178348 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 06:47:43.178398 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 06:47:43.178453 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":120,"threads_started":1}
I20260810 06:47:43.179111 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.173586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:45071 (local address 127.0.0.1:33460)
0810 06:47:43.174413 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.174424 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:43.174489 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:43.174761 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:43.174762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:43.174764 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:43.174809 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.174810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.176308 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.176309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.177400 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.177402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.178813 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.178814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.178874 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.178878 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:47:43.178929 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 06:47:43.179017 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":102,"threads_started":1}
I20260810 06:47:43.179579 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.174092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33460 (local address 127.15.232.194:45071)
0810 06:47:43.174360 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.174365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.174547 (+ 182us) server_negotiation.cc:407] Connection header received
0810 06:47:43.174585 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.174587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.174603 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.174673 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:43.175175 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.176216 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.177496 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.178634 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.178721 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.178781 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.179426 (+ 645us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.179493 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":56,"threads_started":1}
I20260810 06:47:43.180485 31105 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec827ee6708466bb92bf9996f43ea37 (DEFAULT_TABLE table=test-table [id=c5e22232751c4cfebec2491ae34d7e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:43.180895 31105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec827ee6708466bb92bf9996f43ea37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:43.183589 31376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.173404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.15.232.195:45031)
0810 06:47:43.173559 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.173565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.179107 (+ 5542us) server_negotiation.cc:407] Connection header received
0810 06:47:43.179237 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.179241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.179259 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.179313 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:43.179631 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.180654 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.182208 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.183052 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.183121 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.183178 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.183453 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.183504 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":51,"threads_started":1}
I20260810 06:47:43.183864 31377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.173274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:45031 (local address 127.0.0.1:45718)
0810 06:47:43.179043 (+ 5769us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.179054 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:43.179139 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:43.179517 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:43.179518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:43.179519 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:43.179548 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.179549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.180774 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.180775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.182075 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.182078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.183262 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.183263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.183336 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.183340 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:47:43.183727 (+ 387us) client_negotiation.cc:240] Negotiation successful
0810 06:47:43.183786 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5757,"mutex_wait_us":927,"thread_start_us":67,"threads_started":1}
I20260810 06:47:43.184289 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.173920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49344 (local address 127.15.232.193:33229)
0810 06:47:43.174043 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.174047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.174059 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:47:43.174098 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.174101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.174119 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.174438 (+ 319us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:43.174525 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.175616 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.177320 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.178141 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.183999 (+ 5858us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.184091 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.184125 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.184187 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
I20260810 06:47:43.185043 30953 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec827ee6708466bb92bf9996f43ea37 (DEFAULT_TABLE table=test-table [id=c5e22232751c4cfebec2491ae34d7e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:43.185534 30953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec827ee6708466bb92bf9996f43ea37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:43.186077 31261 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec827ee6708466bb92bf9996f43ea37 (DEFAULT_TABLE table=test-table [id=c5e22232751c4cfebec2491ae34d7e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:43.186514 31261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec827ee6708466bb92bf9996f43ea37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:43.187288 31381 tablet_bootstrap.cc:492] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833: Bootstrap starting.
I20260810 06:47:43.188072 31381 tablet_bootstrap.cc:654] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:43.188422 31381 log.cc:824] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:43.189312 31381 tablet_bootstrap.cc:492] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833: No bootstrap required, opened a new log
I20260810 06:47:43.189389 31381 ts_tablet_manager.cc:1397] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:47:43.190001 31383 tablet_bootstrap.cc:492] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1: Bootstrap starting.
I20260810 06:47:43.190747 31383 tablet_bootstrap.cc:654] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:43.191068 31383 log.cc:824] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:43.192750 31381 raft_consensus.cc:348] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d626d15b7104c38918e55312eab75f1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 33229 } } peers { permanent_uuid: "cdb889b2d02946b0836af86d1a484833" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45071 } } peers { permanent_uuid: "f347da363343445787f4134c94cd212f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45031 } }
I20260810 06:47:43.192919 31381 raft_consensus.cc:374] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:43.192986 31381 raft_consensus.cc:729] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdb889b2d02946b0836af86d1a484833, State: Initialized, Role: FOLLOWER
I20260810 06:47:43.193187 31381 consensus_queue.cc:260] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d626d15b7104c38918e55312eab75f1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 33229 } } peers { permanent_uuid: "cdb889b2d02946b0836af86d1a484833" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45071 } } peers { permanent_uuid: "f347da363343445787f4134c94cd212f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45031 } }
I20260810 06:47:43.193506 31381 ts_tablet_manager.cc:1428] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260810 06:47:43.193668 31181 heartbeater.cc:503] Master 127.15.232.254:44293 was elected leader, sending a full tablet report...
I20260810 06:47:43.194978 31383 tablet_bootstrap.cc:492] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1: No bootstrap required, opened a new log
I20260810 06:47:43.195084 31383 ts_tablet_manager.cc:1397] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 06:47:43.197120 31383 raft_consensus.cc:348] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d626d15b7104c38918e55312eab75f1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 33229 } } peers { permanent_uuid: "cdb889b2d02946b0836af86d1a484833" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45071 } } peers { permanent_uuid: "f347da363343445787f4134c94cd212f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45031 } }
I20260810 06:47:43.197384 31383 raft_consensus.cc:374] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:43.197436 31383 raft_consensus.cc:729] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d626d15b7104c38918e55312eab75f1, State: Initialized, Role: FOLLOWER
I20260810 06:47:43.197582 31383 consensus_queue.cc:260] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d626d15b7104c38918e55312eab75f1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 33229 } } peers { permanent_uuid: "cdb889b2d02946b0836af86d1a484833" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45071 } } peers { permanent_uuid: "f347da363343445787f4134c94cd212f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45031 } }
I20260810 06:47:43.197953 31029 heartbeater.cc:503] Master 127.15.232.254:44293 was elected leader, sending a full tablet report...
I20260810 06:47:43.197947 31383 ts_tablet_manager.cc:1428] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:47:43.199184 31384 tablet_bootstrap.cc:492] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f: Bootstrap starting.
I20260810 06:47:43.199993 31384 tablet_bootstrap.cc:654] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:43.200347 31384 log.cc:824] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:43.201372 31384 tablet_bootstrap.cc:492] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f: No bootstrap required, opened a new log
I20260810 06:47:43.201450 31384 ts_tablet_manager.cc:1397] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:47:43.204281 31384 raft_consensus.cc:348] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d626d15b7104c38918e55312eab75f1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 33229 } } peers { permanent_uuid: "cdb889b2d02946b0836af86d1a484833" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45071 } } peers { permanent_uuid: "f347da363343445787f4134c94cd212f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45031 } }
I20260810 06:47:43.204481 31384 raft_consensus.cc:374] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:43.204517 31384 raft_consensus.cc:729] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f347da363343445787f4134c94cd212f, State: Initialized, Role: FOLLOWER
I20260810 06:47:43.204672 31384 consensus_queue.cc:260] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d626d15b7104c38918e55312eab75f1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 33229 } } peers { permanent_uuid: "cdb889b2d02946b0836af86d1a484833" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45071 } } peers { permanent_uuid: "f347da363343445787f4134c94cd212f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45031 } }
I20260810 06:47:43.204972 31384 ts_tablet_manager.cc:1428] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 06:47:43.205016 31341 heartbeater.cc:503] Master 127.15.232.254:44293 was elected leader, sending a full tablet report...
W20260810 06:47:43.217100 31032 tablet.cc:2369] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:47:43.383072 31342 tablet.cc:2369] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:47:43 dist-test-slave-vxj2 krb5kdc[29649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344462, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for HTTP/127.15.232.212@KRBTEST.COM
I20260810 06:47:43.394382 31386 raft_consensus.cc:482] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:47:43.394562 31386 raft_consensus.cc:504] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d626d15b7104c38918e55312eab75f1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 33229 } } peers { permanent_uuid: "cdb889b2d02946b0836af86d1a484833" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45071 } } peers { permanent_uuid: "f347da363343445787f4134c94cd212f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45031 } }
I20260810 06:47:43.395005 31386 leader_election.cc:288] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d626d15b7104c38918e55312eab75f1 (127.15.232.193:33229), f347da363343445787f4134c94cd212f (127.15.232.195:45031)
I20260810 06:47:43.399848 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.395877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:35713 (local address 127.15.232.193:33229)
0810 06:47:43.395921 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.395923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.396249 (+ 326us) server_negotiation.cc:407] Connection header received
0810 06:47:43.396361 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.396362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.396372 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.396404 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:43.396623 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.397371 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.398857 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.399421 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.399473 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.399482 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.399720 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.399766 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:47:43.400169 31397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.395247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:33229 (local address 127.15.232.194:35713)
0810 06:47:43.396163 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.396177 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:43.396263 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:43.396473 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:43.396474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:43.396476 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:43.396555 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.396557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.397476 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.397477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.398733 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.398737 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.399569 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.399570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.399638 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.399642 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:47:43.400014 (+ 372us) client_negotiation.cc:240] Negotiation successful
0810 06:47:43.400067 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"thread_start_us":119,"threads_started":1}
I20260810 06:47:43.400648 30964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec827ee6708466bb92bf9996f43ea37" candidate_uuid: "cdb889b2d02946b0836af86d1a484833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d626d15b7104c38918e55312eab75f1" is_pre_election: true
I20260810 06:47:43.400866 30964 raft_consensus.cc:2454] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdb889b2d02946b0836af86d1a484833 in term 0.
I20260810 06:47:43.401188 31054 leader_election.cc:302] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d626d15b7104c38918e55312eab75f1, cdb889b2d02946b0836af86d1a484833; no voters:
I20260810 06:47:43.401345 31386 raft_consensus.cc:2781] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:47:43.401389 31386 raft_consensus.cc:482] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:47:43.401409 31386 raft_consensus.cc:3037] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:43.402225 31386 raft_consensus.cc:504] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d626d15b7104c38918e55312eab75f1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 33229 } } peers { permanent_uuid: "cdb889b2d02946b0836af86d1a484833" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45071 } } peers { permanent_uuid: "f347da363343445787f4134c94cd212f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45031 } }
I20260810 06:47:43.402419 31386 leader_election.cc:288] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [CANDIDATE]: Term 1 election: Requested vote from peers 9d626d15b7104c38918e55312eab75f1 (127.15.232.193:33229), f347da363343445787f4134c94cd212f (127.15.232.195:45031)
I20260810 06:47:43.402580 30964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec827ee6708466bb92bf9996f43ea37" candidate_uuid: "cdb889b2d02946b0836af86d1a484833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d626d15b7104c38918e55312eab75f1"
I20260810 06:47:43.402657 30964 raft_consensus.cc:3037] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:43.403654 30964 raft_consensus.cc:2454] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdb889b2d02946b0836af86d1a484833 in term 1.
I20260810 06:47:43.403857 31054 leader_election.cc:302] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d626d15b7104c38918e55312eab75f1, cdb889b2d02946b0836af86d1a484833; no voters:
I20260810 06:47:43.403985 31386 raft_consensus.cc:2781] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:47:43.404136 31376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.395751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:37495 (local address 127.15.232.195:45031)
0810 06:47:43.395803 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.395806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.400322 (+ 4516us) server_negotiation.cc:407] Connection header received
0810 06:47:43.400429 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.400431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.400441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.400470 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:43.400629 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.401475 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.403191 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.403744 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.403795 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.403804 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.404023 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.404068 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:47:43.404222 31386 raft_consensus.cc:686] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 1 LEADER]: Becoming Leader. State: Replica: cdb889b2d02946b0836af86d1a484833, State: Running, Role: LEADER
I20260810 06:47:43.404345 31397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.395521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:45031 (local address 127.15.232.194:37495)
0810 06:47:43.400257 (+ 4736us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.400268 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:43.400356 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:43.400531 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:43.400532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:43.400534 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:43.400564 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.400565 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.401569 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.401570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.403083 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.403086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.403887 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.403888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.403946 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.403949 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:47:43.404228 (+ 279us) client_negotiation.cc:240] Negotiation successful
0810 06:47:43.404282 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4714,"thread_start_us":78,"threads_started":1}
I20260810 06:47:43.404335 31386 consensus_queue.cc:237] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d626d15b7104c38918e55312eab75f1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 33229 } } peers { permanent_uuid: "cdb889b2d02946b0836af86d1a484833" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45071 } } peers { permanent_uuid: "f347da363343445787f4134c94cd212f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45031 } }
I20260810 06:47:43.405201 31284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec827ee6708466bb92bf9996f43ea37" candidate_uuid: "cdb889b2d02946b0836af86d1a484833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f347da363343445787f4134c94cd212f" is_pre_election: true
I20260810 06:47:43.405234 30798 catalog_manager.cc:5495] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdb889b2d02946b0836af86d1a484833 (127.15.232.194). New cstate: current_term: 1 leader_uuid: "cdb889b2d02946b0836af86d1a484833" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d626d15b7104c38918e55312eab75f1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 33229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdb889b2d02946b0836af86d1a484833" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 45071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f347da363343445787f4134c94cd212f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45031 } health_report { overall_health: UNKNOWN } } }
I20260810 06:47:43.405385 31284 raft_consensus.cc:2454] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdb889b2d02946b0836af86d1a484833 in term 0.
I20260810 06:47:43.405551 31283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec827ee6708466bb92bf9996f43ea37" candidate_uuid: "cdb889b2d02946b0836af86d1a484833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f347da363343445787f4134c94cd212f"
I20260810 06:47:43.405629 31283 raft_consensus.cc:3037] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:43.406404 31283 raft_consensus.cc:2454] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdb889b2d02946b0836af86d1a484833 in term 1.
W20260810 06:47:43.420975 31184 tablet.cc:2369] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:47:43.464682 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.461463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.15.232.194:45071)
0810 06:47:43.461530 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.461532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.461568 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:47:43.461664 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.461666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.461691 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.461766 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:47:43.462098 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.462649 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.463548 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.463812 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.463899 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.464065 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:47:43.464209 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:47:43.464300 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.464556 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.464583 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:47:43.468005 31283 raft_consensus.cc:1261] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f [term 1 FOLLOWER]: Refusing update from remote peer cdb889b2d02946b0836af86d1a484833: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:47:43.468499 31386 consensus_queue.cc:1035] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f347da363343445787f4134c94cd212f" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:47:43.471525 30964 raft_consensus.cc:1261] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1 [term 1 FOLLOWER]: Refusing update from remote peer cdb889b2d02946b0836af86d1a484833: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:47:43.471966 31386 consensus_queue.cc:1035] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d626d15b7104c38918e55312eab75f1" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 33229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:47:43.474323 31404 mvcc.cc:205] Tried to move back new op lower bound from 7316866922361585664 to 7316866922104909824. Current Snapshot: MvccSnapshot[applied={T|T < 7316866922361585664}]
I20260810 06:47:43.475463 31405 mvcc.cc:205] Tried to move back new op lower bound from 7316866922361585664 to 7316866922104909824. Current Snapshot: MvccSnapshot[applied={T|T < 7316866922361585664}]
I20260810 06:47:43.477723 31407 mvcc.cc:205] Tried to move back new op lower bound from 7316866922361585664 to 7316866922104909824. Current Snapshot: MvccSnapshot[applied={T|T < 7316866922361585664}]
I20260810 06:47:43.484052 30796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:47:43.484192 30796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:47:43.485482 30796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68b3c9f809e14e6da239160c42ef4346: Sending DeleteTablet for 3 replicas of tablet 8ec827ee6708466bb92bf9996f43ea37
I20260810 06:47:43.486229 30953 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ec827ee6708466bb92bf9996f43ea37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:43 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:49344
I20260810 06:47:43.486557 31413 tablet_replica.cc:330] T 8ec827ee6708466bb92bf9996f43ea37 P 9d626d15b7104c38918e55312eab75f1: stopping tablet replica
I20260810 06:47:43.486585 31105 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ec827ee6708466bb92bf9996f43ea37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:43 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:33460
I20260810 06:47:43.486841 31261 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ec827ee6708466bb92bf9996f43ea37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:43 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:45718
I20260810 06:47:43.487623 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 30846
I20260810 06:47:43.500129 31414 tablet_replica.cc:330] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833: stopping tablet replica
I20260810 06:47:43.500322 31414 raft_consensus.cc:2229] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:47:43.500509 31414 raft_consensus.cc:2258] T 8ec827ee6708466bb92bf9996f43ea37 P cdb889b2d02946b0836af86d1a484833 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:47:43.500723 31415 tablet_replica.cc:330] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f: stopping tablet replica
I20260810 06:47:43.500854 31415 raft_consensus.cc:2229] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:47:43.500972 31415 raft_consensus.cc:2258] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 06:47:43.504878 30779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:33229 (error 108)
I20260810 06:47:43.505601 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 31035
I20260810 06:47:43.513926 31377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.513729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:33229 (local address 127.0.0.1:49354)
0810 06:47:43.513805 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.513869 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:33229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:47:43.514142 30779 catalog_manager.cc:4584] TS 9d626d15b7104c38918e55312eab75f1 (127.15.232.193:33229): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ec827ee6708466bb92bf9996f43ea37: Network error: Client connection negotiation failed: client connection to 127.15.232.193:33229: connect: Connection refused (error 111)
I20260810 06:47:43.514529 31415 ts_tablet_manager.cc:1905] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:43.516492 31415 ts_tablet_manager.cc:1918] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:47:43.516587 31415 log.cc:1196] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/8ec827ee6708466bb92bf9996f43ea37
I20260810 06:47:43.516907 31415 ts_tablet_manager.cc:1939] T 8ec827ee6708466bb92bf9996f43ea37 P f347da363343445787f4134c94cd212f: Deleting consensus metadata
I20260810 06:47:43.517613 30781 catalog_manager.cc:4841] TS f347da363343445787f4134c94cd212f (127.15.232.195:45031): tablet 8ec827ee6708466bb92bf9996f43ea37 (table test-table [id=c5e22232751c4cfebec2491ae34d7e88]) successfully deleted
W20260810 06:47:43.530628 31377 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:47:43.529488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:45071 (local address 127.0.0.1:33470)
0810 06:47:43.529543 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.529555 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:43.529599 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:43.530560 (+ 961us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.194:45071: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 06:47:43.531147 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 31183
I20260810 06:47:43.545827 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 30761
I20260810 06:47:43.556200 16291 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 06:47:43.901723 16291 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 06:47:43.901851 16291 mini_ranger.cc:66] Stopping Ranger...
I20260810 06:47:44.247169 16291 mini_ranger.cc:68] Stopped Ranger
2026-08-10 06:47:44.247 UTC [29684] LOG: received smart shutdown request
2026-08-10 06:47:44.252 UTC [29684] LOG: background worker "logical replication launcher" (PID 29693) exited with exit code 1
2026-08-10 06:47:44.252 UTC [29688] LOG: shutting down
2026-08-10 06:47:44.458 UTC [29684] LOG: database system is shut down
2026-08-10T06:47:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52617 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:47:44 dist-test-slave-vxj2 krb5kdc[31426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:47:44 dist-test-slave-vxj2 krb5kdc[31426](info): set up 2 sockets
Aug 10 06:47:44 dist-test-slave-vxj2 krb5kdc[31426](info): commencing operation
krb5kdc: starting...
W20260810 06:47:45.710605 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:47:45 dist-test-slave-vxj2 krb5kdc[31426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:47:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:47:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:47:45.837481 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:36593
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:36593,127.15.232.253:42547,127.15.232.252:39533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:45.923118 31441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:45.923367 31441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:45.923391 31441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:45.924532 31441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:47:45.924571 31441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:45.924584 31441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:45.924597 31441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:47:45.924609 31441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:47:45.926137 31441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:36593,127.15.232.253:42547,127.15.232.252:39533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:36593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:45.926375 31441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:45.926585 31441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:45.932169 31449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:45.932264 31447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:45.932331 31441 server_base.cc:1034] running on GCE node
W20260810 06:47:45.932215 31446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:45.932792 31441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:45.933292 31441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:45.934499 31441 hybrid_clock.cc:648] HybridClock initialized: now 1786344465934482 us; error 32 us; skew 500 ppm
Aug 10 06:47:45 dist-test-slave-vxj2 krb5kdc[31426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344465, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:45.937268 31441 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:47:45.937840 31441 webserver.cc:460] Webserver started at http://127.15.232.254:36293/ using document root <none> and password file <none>
I20260810 06:47:45.938107 31441 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:45.938190 31441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:45.938357 31441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:47:45.939493 31441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "1cc9de575499482a85956222e078a1a8"
format_stamp: "Formatted at 2026-08-10 06:47:45 on dist-test-slave-vxj2"
server_key: "78cd7323264a16df7c59663efed78e3d"
server_key_iv: "56dcd6f9f399da0cee9b13c09dfd8db7"
server_key_version: "encryptionkey@0"
I20260810 06:47:45.939898 31441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "1cc9de575499482a85956222e078a1a8"
format_stamp: "Formatted at 2026-08-10 06:47:45 on dist-test-slave-vxj2"
server_key: "78cd7323264a16df7c59663efed78e3d"
server_key_iv: "56dcd6f9f399da0cee9b13c09dfd8db7"
server_key_version: "encryptionkey@0"
I20260810 06:47:45.941397 31441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:47:45.942240 31457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:45.942420 31441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:47:45.942533 31441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "1cc9de575499482a85956222e078a1a8"
format_stamp: "Formatted at 2026-08-10 06:47:45 on dist-test-slave-vxj2"
server_key: "78cd7323264a16df7c59663efed78e3d"
server_key_iv: "56dcd6f9f399da0cee9b13c09dfd8db7"
server_key_version: "encryptionkey@0"
I20260810 06:47:45.942618 31441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:45.963039 31441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:45.965261 31441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:45.965458 31441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:45.969691 31441 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:36593
I20260810 06:47:45.969743 31518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:36593 every 8 connection(s)
I20260810 06:47:45.970158 31441 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:47:45.971082 31519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:45.972668 31519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:47:45.973196 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 31441
I20260810 06:47:45.973351 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:47:45.973578 16291 external_mini_cluster.cc:1428] Setting key 52e759090c603cf556734c14d4fda417
Aug 10 06:47:45 dist-test-slave-vxj2 krb5kdc[31426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344465, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.253@KRBTEST.COM; defaulting to no policy
I20260810 06:47:45.985224 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:45.973593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:36593 (local address 127.0.0.1:48018)
0810 06:47:45.973781 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:45.973836 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:45.974594 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:45.974939 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:45.974942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:45.975353 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:45.975542 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:45.975546 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:45.977390 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:45.977391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:45.978210 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:45.978213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:45.979063 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:45.979064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:45.979134 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:45.979774 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:45.979784 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:45.981573 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:45.983526 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:45.983534 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:45.983536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:45.983820 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:45.984066 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:45.984066 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:45.984067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:45.984157 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:45.984543 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:45.984548 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:45.984609 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 06:47:45.984689 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:47:45.984822 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260810 06:47:45.985248 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:45.973597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48018 (local address 127.15.232.254:36593)
0810 06:47:45.973851 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:45.973855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:45.973951 (+ 96us) server_negotiation.cc:407] Connection header received
0810 06:47:45.974776 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:45.974778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:45.974797 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:45.974895 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:45.975677 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:45.977283 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:45.978322 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:45.978947 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:45.979094 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:45.981676 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:45.981685 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:45.981685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:45.981692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:45.983410 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:45.983893 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:45.983896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:45.983897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:45.983996 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:45.984216 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:45.984217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:45.984217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:45.984477 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:45.984545 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:45.984860 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 06:47:45.984949 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":94,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.253@KRBTEST.COM" created.
I20260810 06:47:45.986060 31519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:47:45.986503 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:45.986342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:42547 (local address 127.0.0.1:46490)
0810 06:47:45.986399 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:45.986466 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 06:47:45.986671 31462 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.253:42547: connect: Connection refused (error 111)
I20260810 06:47:45.987162 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:45.987052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:42547 (local address 127.0.0.1:46504)
0810 06:47:45.987102 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:45.987133 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 06:47:45.987293 31519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.253:42547: Network error: Client connection negotiation failed: client connection to 127.15.232.253:42547: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.253.keytab.
Entry for principal kudu/127.15.232.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.253.keytab.
I20260810 06:47:46.005923 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.253:42547
--webserver_interface=127.15.232.253
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.253.keytab
--principal=kudu/127.15.232.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:36593,127.15.232.253:42547,127.15.232.252:39533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:47:46.036582 31519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } attempt: 1
I20260810 06:47:46.037288 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.037090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:42547 (local address 127.0.0.1:46514)
0810 06:47:46.037202 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.037252 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 06:47:46.037746 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.037658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:42547 (local address 127.0.0.1:46526)
0810 06:47:46.037702 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.037729 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 06:47:46.037863 31519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.253:42547: Network error: Client connection negotiation failed: client connection to 127.15.232.253:42547: connect: Connection refused (error 111)
W20260810 06:47:46.095201 31529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:46.095465 31529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:46.095492 31529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:46.096784 31529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:47:46.096848 31529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:46.096864 31529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:46.096879 31529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:47:46.096894 31529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:47:46.098563 31529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:36593,127.15.232.253:42547,127.15.232.252:39533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.253.keytab
--principal=kudu/127.15.232.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.253:42547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.232.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:46.098881 31529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:46.099167 31529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:46.104537 31536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:46.104750 31529 server_base.cc:1034] running on GCE node
W20260810 06:47:46.104525 31538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:46.104537 31535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:46.105255 31529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:46.105590 31529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:46.106038 31519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } attempt: 2
I20260810 06:47:46.106602 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.106425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:42547 (local address 127.0.0.1:46532)
0810 06:47:46.106527 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.106566 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 06:47:46.106803 31529 hybrid_clock.cc:648] HybridClock initialized: now 1786344466106766 us; error 50 us; skew 500 ppm
I20260810 06:47:46.107142 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.107013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:42547 (local address 127.0.0.1:46534)
0810 06:47:46.107069 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.107101 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:47:46.107270 31519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.253:42547: Network error: Client connection negotiation failed: client connection to 127.15.232.253:42547: connect: Connection refused (error 111)
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344466, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:46.109824 31529 init.cc:377] Logged in from keytab as kudu/127.15.232.253@KRBTEST.COM (short username kudu)
I20260810 06:47:46.110271 31529 webserver.cc:460] Webserver started at http://127.15.232.253:46087/ using document root <none> and password file <none>
I20260810 06:47:46.110529 31529 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:46.110579 31529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:46.110745 31529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:47:46.111910 31529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/data/instance:
uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f"
format_stamp: "Formatted at 2026-08-10 06:47:46 on dist-test-slave-vxj2"
server_key: "dd967274af006b76cb87570c34bfcf99"
server_key_iv: "5d74e4c2c6fa44e674410579ef3d8c2b"
server_key_version: "encryptionkey@0"
I20260810 06:47:46.112305 31529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/wal/instance:
uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f"
format_stamp: "Formatted at 2026-08-10 06:47:46 on dist-test-slave-vxj2"
server_key: "dd967274af006b76cb87570c34bfcf99"
server_key_iv: "5d74e4c2c6fa44e674410579ef3d8c2b"
server_key_version: "encryptionkey@0"
I20260810 06:47:46.113960 31529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:47:46.114926 31546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:46.115177 31529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:46.115263 31529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/wal
uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f"
format_stamp: "Formatted at 2026-08-10 06:47:46 on dist-test-slave-vxj2"
server_key: "dd967274af006b76cb87570c34bfcf99"
server_key_iv: "5d74e4c2c6fa44e674410579ef3d8c2b"
server_key_version: "encryptionkey@0"
I20260810 06:47:46.115361 31529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:46.129927 31529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:46.132485 31529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:46.132660 31529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:46.137081 31529 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.253:42547
I20260810 06:47:46.137217 31605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.253:42547 every 8 connection(s)
I20260810 06:47:46.137567 31529 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
I20260810 06:47:46.138491 31606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:46.140157 31606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:47:46.142092 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 31529
I20260810 06:47:46.142232 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-1/wal/instance
I20260810 06:47:46.142421 16291 external_mini_cluster.cc:1428] Setting key f7bc585e852a415ce1ad7d261e95e5b3
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344466, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.253@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:46.151368 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.140917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48026 (local address 127.15.232.254:36593)
0810 06:47:46.141017 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.141019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.141199 (+ 180us) server_negotiation.cc:407] Connection header received
0810 06:47:46.141714 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.141715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.141729 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.141792 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.142534 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.143293 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.144906 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.145241 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.145320 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.147902 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.147904 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.147905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.147914 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.149790 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.150226 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.150228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.150228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.150297 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.150582 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.150583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.150584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.150735 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.150815 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.151131 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.151228 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:47:46.151381 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.140873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:36593 (local address 127.0.0.1:48026)
0810 06:47:46.141038 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.141080 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:46.141627 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:46.141844 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:46.141847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:46.142126 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:46.142401 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.142406 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.143428 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.143429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.144796 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.144799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.145364 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.145366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.145543 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.146147 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:46.146157 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:46.147799 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:46.149884 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.149892 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.149893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.150139 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.150392 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.150393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.150393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.150503 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.150841 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:46.150848 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:46.150979 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 06:47:46.151079 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:47:46.151211 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for kudu/127.15.232.252@KRBTEST.COM; defaulting to no policy
I20260810 06:47:46.152338 31606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.252@KRBTEST.COM" created.
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344466, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.253@KRBTEST.COM for kudu/127.15.232.253@KRBTEST.COM
I20260810 06:47:46.161012 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.152706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.15.232.253:42547)
0810 06:47:46.152877 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.152900 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.152925 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:47:46.152948 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.152950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.152964 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.153067 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.153387 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.154107 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.155108 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.155513 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.155594 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.157606 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.157616 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.157616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.157624 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.159632 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.159985 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.159989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.159989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.160068 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.160303 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.160305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.160305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.160483 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.160559 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.160753 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.160903 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":145,"thread_start_us":97,"threads_started":1}
I20260810 06:47:46.161038 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.152639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:42547 (local address 127.0.0.1:46548)
0810 06:47:46.152708 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.152717 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:46.152747 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:46.153083 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:46.153084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:46.153264 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:46.153316 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.153317 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.154246 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.154247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.154985 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.154989 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.155607 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.155608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.155682 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.156190 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:46.156194 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:46.157497 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:46.159726 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.159727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.159727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.159893 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.160158 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.160159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.160160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.160228 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.160567 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:46.160568 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:46.160625 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 06:47:46.160714 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:47:46.160833 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 06:47:46.161873 31606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:47:46.162284 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.162136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34776)
0810 06:47:46.162193 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.162252 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:47:46.162444 31548 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
I20260810 06:47:46.162997 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.162881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34784)
0810 06:47:46.162937 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.162969 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 06:47:46.163152 31606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:39533: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.252.keytab.
Entry for principal kudu/127.15.232.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.252.keytab.
I20260810 06:47:46.173118 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.252:39533
--webserver_interface=127.15.232.252
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.252.keytab
--principal=kudu/127.15.232.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:36593,127.15.232.253:42547,127.15.232.252:39533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:47:46.198446 31519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } attempt: 3
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344465, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.253@KRBTEST.COM
I20260810 06:47:46.206007 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.198870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:42547 (local address 127.0.0.1:46562)
0810 06:47:46.198939 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.198953 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:46.199009 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:46.199156 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:46.199157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:46.199338 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:46.199408 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.199410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.200138 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.200139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.200664 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.200666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.201063 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.201064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.201204 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.201693 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:46.201706 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:46.203186 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:46.204879 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.204880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.204880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.205068 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.205309 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.205310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.205311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.205375 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.205649 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:46.205650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:46.205700 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 06:47:46.205783 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:47:46.205899 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 06:47:46.205998 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.198964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46562 (local address 127.15.232.253:42547)
0810 06:47:46.199025 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.199027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.199039 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:47:46.199070 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.199071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.199084 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.199152 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.199527 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.200041 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.200760 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.200983 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.201066 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.203281 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.203282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.203283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.203291 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.204766 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.205143 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.205145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.205145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.205232 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.205447 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.205448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.205448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.205571 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.205646 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.205809 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.205899 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 06:47:46.206583 31519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:47:46.207048 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.206916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34798)
0810 06:47:46.206977 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.207010 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:47:46.207458 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.207362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34800)
0810 06:47:46.207410 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.207440 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:47:46.207576 31519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:39533: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
I20260810 06:47:46.212428 31606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } attempt: 1
I20260810 06:47:46.212886 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.212722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34808)
0810 06:47:46.212795 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.212843 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 06:47:46.213340 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.213225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34820)
0810 06:47:46.213276 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.213310 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 06:47:46.213474 31606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:39533: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
I20260810 06:47:46.238729 31519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } attempt: 1
I20260810 06:47:46.239306 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.239154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34828)
0810 06:47:46.239228 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.239277 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 06:47:46.239696 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.239616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34840)
0810 06:47:46.239655 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.239680 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 06:47:46.239823 31519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:39533: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
W20260810 06:47:46.260684 31613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:46.260974 31613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:46.261019 31613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:46.262298 31613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:47:46.262382 31613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:46.262408 31613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:46.262442 31613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:47:46.262470 31613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:47:46.264128 31613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:36593,127.15.232.253:42547,127.15.232.252:39533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.252.keytab
--principal=kudu/127.15.232.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.252:39533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.232.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:46.264465 31613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:46.264719 31613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:46.270176 31618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:46.270138 31620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:46.270133 31622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:46.270397 31613 server_base.cc:1034] running on GCE node
I20260810 06:47:46.270655 31613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:46.270923 31613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:46.272073 31613 hybrid_clock.cc:648] HybridClock initialized: now 1786344466272058 us; error 40 us; skew 500 ppm
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344466, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:46.274755 31613 init.cc:377] Logged in from keytab as kudu/127.15.232.252@KRBTEST.COM (short username kudu)
I20260810 06:47:46.275259 31613 webserver.cc:460] Webserver started at http://127.15.232.252:35957/ using document root <none> and password file <none>
I20260810 06:47:46.275466 31613 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:46.275508 31613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:46.275604 31613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:47:46.276661 31613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/data/instance:
uuid: "c72743dce8224acab91ec84df7d29861"
format_stamp: "Formatted at 2026-08-10 06:47:46 on dist-test-slave-vxj2"
server_key: "28f0394ec3067c97961b45c4760c6561"
server_key_iv: "2798a8113b1b1180006cb4eca64f6708"
server_key_version: "encryptionkey@0"
I20260810 06:47:46.276978 31613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/wal/instance:
uuid: "c72743dce8224acab91ec84df7d29861"
format_stamp: "Formatted at 2026-08-10 06:47:46 on dist-test-slave-vxj2"
server_key: "28f0394ec3067c97961b45c4760c6561"
server_key_iv: "2798a8113b1b1180006cb4eca64f6708"
server_key_version: "encryptionkey@0"
I20260810 06:47:46.278380 31613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:47:46.279074 31629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:46.279251 31613 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:47:46.279345 31613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/wal
uuid: "c72743dce8224acab91ec84df7d29861"
format_stamp: "Formatted at 2026-08-10 06:47:46 on dist-test-slave-vxj2"
server_key: "28f0394ec3067c97961b45c4760c6561"
server_key_iv: "2798a8113b1b1180006cb4eca64f6708"
server_key_version: "encryptionkey@0"
I20260810 06:47:46.279402 31613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:46.281647 31606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } attempt: 2
I20260810 06:47:46.282070 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.281939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34846)
0810 06:47:46.282002 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.282040 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 06:47:46.282455 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.282368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34854)
0810 06:47:46.282412 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.282437 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 06:47:46.282572 31606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:39533: Network error: Client connection negotiation failed: client connection to 127.15.232.252:39533: connect: Connection refused (error 111)
I20260810 06:47:46.304771 31613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:46.307301 31613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:46.307475 31613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:46.311694 31613 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.252:39533
I20260810 06:47:46.311789 31690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.252:39533 every 8 connection(s)
I20260810 06:47:46.312155 31613 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
I20260810 06:47:46.313145 31691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:46.314962 31519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } attempt: 2
I20260810 06:47:46.314851 31691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:47:46.318859 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 31613
I20260810 06:47:46.319003 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344303981681-16291-0/minicluster-data/master-2/wal/instance
I20260810 06:47:46.319212 16291 external_mini_cluster.cc:1428] Setting key 02da1364e92c56bdbc316fee5c264f4b
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344465, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.252@KRBTEST.COM
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344466, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.252@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.252@KRBTEST.COM
I20260810 06:47:46.325894 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.315290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34856)
0810 06:47:46.315353 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.315368 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:46.315416 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:46.316448 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:46.316449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:46.316649 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:46.316709 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.316711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.318187 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.318189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.318723 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.318725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.319615 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.319616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.319684 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.320267 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:46.320272 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:46.322278 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:46.324548 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.324550 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.324550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.324784 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.325012 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.325013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.325013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.325083 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.325478 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:46.325479 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:46.325527 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:47:46.325600 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:47:46.325778 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":70}
I20260810 06:47:46.326102 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.315579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48042 (local address 127.15.232.254:36593)
0810 06:47:46.315617 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.315619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.315827 (+ 208us) server_negotiation.cc:407] Connection header received
0810 06:47:46.316366 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.316367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.316379 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.316443 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.317048 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.317836 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.318866 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.319761 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.319844 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.322522 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.322523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.322523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.322531 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.324066 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.324600 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.324601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.324602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.324654 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.324971 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.324973 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.324973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.325212 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.325274 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.325590 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.325710 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:47:46.326161 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.315435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34856 (local address 127.15.232.252:39533)
0810 06:47:46.315686 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.315703 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.315729 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:47:46.316228 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.316238 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.316261 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.316395 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.316803 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.318103 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.318810 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.319352 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.319495 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.322362 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.322370 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.322370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.322377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.324447 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.324867 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.324871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.324871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.324941 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.325143 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.325143 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.325144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.325394 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.325497 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.325709 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.325866 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":86,"threads_started":1}
I20260810 06:47:46.327080 31694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.315536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:36593 (local address 127.0.0.1:48042)
0810 06:47:46.315688 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.315729 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:46.316250 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:46.316476 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:46.316478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:46.316831 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:46.316960 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.316965 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.317928 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.317929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.318795 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.318798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.319852 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.319853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.319937 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.320449 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:46.320461 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:46.322434 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:46.324174 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.324181 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.324183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.324535 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.324729 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.324730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.324730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.324877 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.325334 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:46.325341 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:46.325419 (+ 78us) client_negotiation.cc:769] Sending connection context
0810 06:47:46.325796 (+ 377us) client_negotiation.cc:240] Negotiation successful
0810 06:47:46.326560 (+ 764us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260810 06:47:46.328243 31519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8: Bootstrap starting.
I20260810 06:47:46.328990 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.319643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34864 (local address 127.15.232.252:39533)
0810 06:47:46.319994 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.319996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.320004 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:47:46.320023 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.320024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.320034 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.320141 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.320909 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.321445 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.322276 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.322599 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.322671 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.325574 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.325575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.325575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.325581 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.327722 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.328136 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.328137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.328138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.328199 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.328410 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.328411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.328411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.328547 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.328693 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.328789 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.328868 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":85,"threads_started":1}
I20260810 06:47:46.329123 31519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:46.329531 31519 log.cc:824] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:46.329619 31691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:47:46.330662 31519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8: No bootstrap required, opened a new log
I20260810 06:47:46.332872 31519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } }
I20260810 06:47:46.333094 31519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:46.333175 31519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc9de575499482a85956222e078a1a8, State: Initialized, Role: FOLLOWER
I20260810 06:47:46.333348 31519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } }
I20260810 06:47:46.333736 31519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:47:46.333678 31698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } } }
I20260810 06:47:46.333863 31698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [sys.catalog]: This master's current role is: FOLLOWER
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344466, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.252@KRBTEST.COM for kudu/127.15.232.253@KRBTEST.COM
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
W20260810 06:47:46.335801 31709 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:47:46.335898 31709 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:47:46.338696 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.330018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46576 (local address 127.15.232.253:42547)
0810 06:47:46.330095 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.330097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.330106 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:47:46.330131 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.330132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.330141 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.330191 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.330581 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.331258 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.332217 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.332524 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.332569 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.335566 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.335568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.335568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.335577 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.337221 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.337750 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.337751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.337752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.337829 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.338045 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.338046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.338047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.338187 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.338230 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.338509 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.338605 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:47:46.339195 31694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.329957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:42547 (local address 127.0.0.1:46576)
0810 06:47:46.329988 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.329998 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:46.330058 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:46.330244 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:46.330245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:46.330406 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:46.330480 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.330482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.331364 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.331365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.332111 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.332114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.332653 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.332654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.332724 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.333221 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:46.333226 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:46.335453 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:46.337342 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.337343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.337344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.337613 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.337910 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.337910 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.337911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.337975 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.338277 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:46.338278 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:46.338364 (+ 86us) client_negotiation.cc:769] Sending connection context
0810 06:47:46.338991 (+ 627us) client_negotiation.cc:240] Negotiation successful
0810 06:47:46.339100 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 06:47:46.339465 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.329881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48050 (local address 127.15.232.254:36593)
0810 06:47:46.329938 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.329940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.330099 (+ 159us) server_negotiation.cc:407] Connection header received
0810 06:47:46.330189 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.330190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.330200 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.330249 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.330734 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.331210 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.331921 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.332117 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.332192 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.335627 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.335628 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.335628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.335635 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.337877 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.338296 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.338297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.338297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.338367 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.338593 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.338593 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.338594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.338722 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.338884 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.339291 (+ 407us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.339381 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:47:46.339860 31691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:47:46.336685 31441 master_runner.cc:186] Error getting master registration for 127.15.232.253:42547: OK, instance_id { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" instance_seqno: 1786344466135850 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:47:46.340093 31441 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344466, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.252@KRBTEST.COM for kudu/127.15.232.252@KRBTEST.COM
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.253@KRBTEST.COM
I20260810 06:47:46.348603 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.340428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46580 (local address 127.15.232.253:42547)
0810 06:47:46.340469 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.340471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.340479 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:47:46.340498 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.340499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.340507 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.340556 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.341231 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.341831 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.342501 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.342702 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.342782 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.345844 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.345846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.345846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.345855 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.347366 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.347736 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.347737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.347738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.347795 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.348014 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.348015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.348016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.348142 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.348342 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.348379 (+ 37us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.348480 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:47:46.349586 31694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.340209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34866)
0810 06:47:46.340285 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.340296 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:46.340348 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:46.340598 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:46.340599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:46.340718 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:46.340748 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.340750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.341627 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.341628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.342427 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.342430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.342916 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.342917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.342979 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.343535 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:46.343540 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:46.345702 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:46.348215 (+ 2513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.348216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.348216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.348410 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.348589 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.348590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.348590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.348636 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.349282 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:46.349282 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:46.349323 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 06:47:46.349386 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 06:47:46.349483 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 06:47:46.349920 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.340249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.15.232.252:39533)
0810 06:47:46.340288 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.340290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.340333 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:47:46.340405 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.340405 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.340416 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.340460 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.340847 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.341533 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.342522 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.342835 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.342903 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.345819 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.345821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.345821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.345830 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.348105 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.348481 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.348482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.348482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.348530 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.348853 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.348854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.348854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.349126 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.349258 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.349762 (+ 504us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.349845 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:47:46.350106 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:47:46.351477 31691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861: Bootstrap starting.
I20260810 06:47:46.352385 31691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:46.352717 31691 log.cc:824] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:46.353554 31691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861: No bootstrap required, opened a new log
I20260810 06:47:46.355080 31691 raft_consensus.cc:348] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } }
I20260810 06:47:46.355239 31691 raft_consensus.cc:374] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:46.355305 31691 raft_consensus.cc:729] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c72743dce8224acab91ec84df7d29861, State: Initialized, Role: FOLLOWER
I20260810 06:47:46.355448 31691 consensus_queue.cc:260] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } }
I20260810 06:47:46.355722 31720 sys_catalog.cc:455] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } } }
I20260810 06:47:46.355789 31691 sys_catalog.cc:564] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:47:46.355872 31720 sys_catalog.cc:458] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:47:46.358139 31735 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:47:46.358248 31735 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:47:46.359912 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.351648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.15.232.253:42547)
0810 06:47:46.351688 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.351690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.352079 (+ 389us) server_negotiation.cc:407] Connection header received
0810 06:47:46.352174 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.352175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.352185 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.352214 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.352983 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.353788 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.354873 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.355161 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.355202 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.356800 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.356801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.356802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.356809 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.358495 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.358990 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.358991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.358991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.359049 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.359252 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.359253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.359253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.359383 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.359454 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.359725 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.359817 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:47:46.359611 31613 master_runner.cc:186] Error getting master registration for 127.15.232.253:42547: OK, instance_id { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" instance_seqno: 1786344466135850 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:47:46.361955 31613 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:47:46.368806 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.351743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48062 (local address 127.15.232.254:36593)
0810 06:47:46.351797 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.351799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.360831 (+ 9032us) server_negotiation.cc:407] Connection header received
0810 06:47:46.360929 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.360930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.360941 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.360972 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.361661 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.362413 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.363423 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.363738 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.363823 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.365382 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.365384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.365384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.365392 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.367128 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.367565 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.367567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.367567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.367632 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.367880 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.367881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.367882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.368025 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.368140 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.368342 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.368427 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:47:46.372429 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.352104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34870 (local address 127.15.232.252:39533)
0810 06:47:46.352138 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.352139 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.365356 (+ 13217us) server_negotiation.cc:407] Connection header received
0810 06:47:46.365459 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.365460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.365471 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.365503 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.366163 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.366953 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.367858 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.368153 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.368212 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.369609 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.369611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.369611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.369618 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.371199 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.371612 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.371613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.371613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.371676 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.371889 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.371890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.371890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.372022 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.372138 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.372253 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.372334 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 06:47:46.373729 31606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } attempt: 3
Aug 10 06:47:46 dist-test-slave-vxj2 krb5kdc[31426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344466, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.253@KRBTEST.COM for kudu/127.15.232.252@KRBTEST.COM
I20260810 06:47:46.383764 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.374097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34886 (local address 127.15.232.252:39533)
0810 06:47:46.374200 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:46.374202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:46.374210 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:47:46.374228 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:46.374229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:46.374238 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:46.374286 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:46.374630 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.375839 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.376855 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.377224 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.377300 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.379721 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:46.379723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:46.379723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:46.379732 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:46.382297 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.382718 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.382719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.382719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.382788 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:46.383077 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:46.383078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:46.383078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:46.383227 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:46.383420 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:46.383578 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 06:47:46.383664 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 06:47:46.383764 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:46.374026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:39533 (local address 127.0.0.1:34886)
0810 06:47:46.374081 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:46.374092 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:46.374129 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:46.374301 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:46.374303 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:46.374491 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:46.374545 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.374547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.375931 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.375932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.376749 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:46.376752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:46.377326 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:46.377327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:46.377393 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:46.377947 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:46.377953 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:46.379622 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:46.382374 (+ 2752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.382375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.382376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.382636 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.382925 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:46.382926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:46.382927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:46.383004 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:46.383391 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:46.383392 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:46.383458 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 06:47:46.383537 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:47:46.383653 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:47:46.386139 31606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f: Bootstrap starting.
I20260810 06:47:46.387015 31606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:46.387354 31606 log.cc:824] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:46.388890 31606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f: No bootstrap required, opened a new log
I20260810 06:47:46.391311 31606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } }
I20260810 06:47:46.391577 31606 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:46.391625 31606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e31ad2bc1c04323ac8b7aa949d5df6f, State: Initialized, Role: FOLLOWER
I20260810 06:47:46.391830 31606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } }
I20260810 06:47:46.392259 31606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:47:46.392220 31740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } } }
I20260810 06:47:46.392441 31740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:47:46.394963 31755 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:47:46.395109 31755 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:47:46.397302 31529 master_runner.cc:263] No leader master found from master 5e31ad2bc1c04323ac8b7aa949d5df6f
I20260810 06:47:46.397399 31529 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:47:46.463797 31740 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:47:46.463956 31740 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } }
I20260810 06:47:46.464207 31740 leader_election.cc:288] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc9de575499482a85956222e078a1a8 (127.15.232.254:36593), c72743dce8224acab91ec84df7d29861 (127.15.232.252:39533)
I20260810 06:47:46.464757 31660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c72743dce8224acab91ec84df7d29861" is_pre_election: true
I20260810 06:47:46.464938 31660 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e31ad2bc1c04323ac8b7aa949d5df6f in term 0.
I20260810 06:47:46.465266 31491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc9de575499482a85956222e078a1a8" is_pre_election: true
I20260810 06:47:46.465440 31491 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e31ad2bc1c04323ac8b7aa949d5df6f in term 0.
I20260810 06:47:46.465804 31548 leader_election.cc:302] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e31ad2bc1c04323ac8b7aa949d5df6f, c72743dce8224acab91ec84df7d29861; no voters:
I20260810 06:47:46.466114 31740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:47:46.466171 31740 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:47:46.466199 31740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [term 0 FOLLOWER]: Advancing to term 1
W20260810 06:47:46.466315 31464 tablet.cc:2369] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:47:46.467229 31740 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } }
I20260810 06:47:46.467378 31740 leader_election.cc:288] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [CANDIDATE]: Term 1 election: Requested vote from peers 1cc9de575499482a85956222e078a1a8 (127.15.232.254:36593), c72743dce8224acab91ec84df7d29861 (127.15.232.252:39533)
I20260810 06:47:46.467803 31491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc9de575499482a85956222e078a1a8"
I20260810 06:47:46.467886 31491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:46.468856 31491 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31ad2bc1c04323ac8b7aa949d5df6f in term 1.
I20260810 06:47:46.469122 31660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c72743dce8224acab91ec84df7d29861"
I20260810 06:47:46.469254 31660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:46.470253 31660 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31ad2bc1c04323ac8b7aa949d5df6f in term 1.
I20260810 06:47:46.470551 31548 leader_election.cc:302] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e31ad2bc1c04323ac8b7aa949d5df6f, c72743dce8224acab91ec84df7d29861; no voters:
I20260810 06:47:46.470713 31740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:47:46.470890 31740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [term 1 LEADER]: Becoming Leader. State: Replica: 5e31ad2bc1c04323ac8b7aa949d5df6f, State: Running, Role: LEADER
I20260810 06:47:46.470988 31740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } }
I20260810 06:47:46.471424 31740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e31ad2bc1c04323ac8b7aa949d5df6f. Latest consensus state: current_term: 1 leader_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } } }
I20260810 06:47:46.471496 31740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:46.471843 31759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:47:46.472074 31759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:47:46.474500 31660 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [term 1 FOLLOWER]: Refusing update from remote peer 5e31ad2bc1c04323ac8b7aa949d5df6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:47:46.474824 31491 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [term 1 FOLLOWER]: Refusing update from remote peer 5e31ad2bc1c04323ac8b7aa949d5df6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:47:46.476183 31740 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:47:46.476730 31740 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:47:46.478261 31720 sys_catalog.cc:455] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e31ad2bc1c04323ac8b7aa949d5df6f. Latest consensus state: current_term: 1 leader_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } } }
I20260810 06:47:46.478396 31720 sys_catalog.cc:458] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:47:46.479410 31698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e31ad2bc1c04323ac8b7aa949d5df6f. Latest consensus state: current_term: 1 leader_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } } }
I20260810 06:47:46.479502 31698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:47:46.481240 31740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } } }
I20260810 06:47:46.481365 31740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:46.481932 31698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } } }
I20260810 06:47:46.482019 31698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:47:46.482344 31760 mvcc.cc:205] Tried to move back new op lower bound from 7316866934676140032 to 7316866934665695232. Current Snapshot: MvccSnapshot[applied={T|T < 7316866934676140032}]
I20260810 06:47:46.482602 31763 mvcc.cc:205] Tried to move back new op lower bound from 7316866934676140032 to 7316866934665695232. Current Snapshot: MvccSnapshot[applied={T|T < 7316866934676140032}]
I20260810 06:47:46.483637 31740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } } }
I20260810 06:47:46.483721 31740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:46.484172 31720 sys_catalog.cc:455] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc9de575499482a85956222e078a1a8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36593 } } peers { permanent_uuid: "5e31ad2bc1c04323ac8b7aa949d5df6f" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 42547 } } peers { permanent_uuid: "c72743dce8224acab91ec84df7d29861" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 39533 } } }
I20260810 06:47:46.484266 31720 sys_catalog.cc:458] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:47:46.485018 31761 mvcc.cc:205] Tried to move back new op lower bound from 7316866934676140032 to 7316866934665695232. Current Snapshot: MvccSnapshot[applied={T|T < 7316866934676140032}]
I20260810 06:47:46.485684 31759 catalog_manager.cc:1347] Generated new cluster ID: d639c321ae1544fd8fbd687ab6dd862f
I20260810 06:47:46.485733 31759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:47:46.520810 31759 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:47:46.521592 31759 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:47:46.524899 31759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e31ad2bc1c04323ac8b7aa949d5df6f: Generated new TSK 0
I20260810 06:47:46.525141 31759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:47:47.336397 31709 catalog_manager.cc:1259] Loaded cluster ID: d639c321ae1544fd8fbd687ab6dd862f
I20260810 06:47:47.336515 31709 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8: loading cluster ID for follower catalog manager: success
I20260810 06:47:47.337456 31709 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8: acquiring CA information for follower catalog manager: success
I20260810 06:47:47.337796 31709 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1cc9de575499482a85956222e078a1a8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:47:47.358770 31735 catalog_manager.cc:1259] Loaded cluster ID: d639c321ae1544fd8fbd687ab6dd862f
I20260810 06:47:47.358858 31735 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861: loading cluster ID for follower catalog manager: success
I20260810 06:47:47.359720 31735 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861: acquiring CA information for follower catalog manager: success
I20260810 06:47:47.360080 31735 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c72743dce8224acab91ec84df7d29861: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:47:47.578392 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 31441
I20260810 06:47:47.586207 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 31529
I20260810 06:47:47.593508 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 31613
2026-08-10T06:47:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3020 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:47:47 dist-test-slave-vxj2 krb5kdc[31776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:47:47 dist-test-slave-vxj2 krb5kdc[31776](info): set up 2 sockets
Aug 10 06:47:47 dist-test-slave-vxj2 krb5kdc[31776](info): commencing operation
krb5kdc: starting...
W20260810 06:47:48.770246 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:47:48 dist-test-slave-vxj2 krb5kdc[31776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:47:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:47:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:47:48.896617 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:42845
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:42845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:48.980201 31789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:48.980450 31789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:48.980474 31789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:48.981642 31789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:47:48.981686 31789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:48.981700 31789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:48.981714 31789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:47:48.981725 31789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:47:48.983211 31789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:42845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:42845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:48.983428 31789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:48.983608 31789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:48.988679 31795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:48.988679 31796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:48.988824 31789 server_base.cc:1034] running on GCE node
W20260810 06:47:48.988682 31798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:48.989281 31789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:48.989621 31789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:48.990800 31789 hybrid_clock.cc:648] HybridClock initialized: now 1786344468990787 us; error 36 us; skew 500 ppm
Aug 10 06:47:48 dist-test-slave-vxj2 krb5kdc[31776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344468, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:48.993470 31789 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:47:48.993911 31789 webserver.cc:460] Webserver started at http://127.15.232.254:44175/ using document root <none> and password file <none>
I20260810 06:47:48.994159 31789 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:48.994207 31789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:48.994362 31789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:47:48.995483 31789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "6e25b60e80564643a1cf6144d69927b1"
format_stamp: "Formatted at 2026-08-10 06:47:48 on dist-test-slave-vxj2"
server_key: "0f3764b4d8e0dc8042a2c122b47fe8c0"
server_key_iv: "bd2be0fb59af2488c3fae654881ffc13"
server_key_version: "encryptionkey@0"
I20260810 06:47:48.995872 31789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "6e25b60e80564643a1cf6144d69927b1"
format_stamp: "Formatted at 2026-08-10 06:47:48 on dist-test-slave-vxj2"
server_key: "0f3764b4d8e0dc8042a2c122b47fe8c0"
server_key_iv: "bd2be0fb59af2488c3fae654881ffc13"
server_key_version: "encryptionkey@0"
I20260810 06:47:48.997360 31789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260810 06:47:48.998165 31805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:48.998343 31789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:48.998469 31789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "6e25b60e80564643a1cf6144d69927b1"
format_stamp: "Formatted at 2026-08-10 06:47:48 on dist-test-slave-vxj2"
server_key: "0f3764b4d8e0dc8042a2c122b47fe8c0"
server_key_iv: "bd2be0fb59af2488c3fae654881ffc13"
server_key_version: "encryptionkey@0"
I20260810 06:47:48.998551 31789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:49.009891 31789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:49.012005 31789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:49.012184 31789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:49.016413 31789 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:42845
I20260810 06:47:49.016448 31866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:42845 every 8 connection(s)
I20260810 06:47:49.016875 31789 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:47:49.017812 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:49.020205 31868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1: Bootstrap starting.
I20260810 06:47:49.020800 31868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:49.021100 31868 log.cc:824] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:49.022075 31868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1: No bootstrap required, opened a new log
I20260810 06:47:49.022294 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 31789
I20260810 06:47:49.022403 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:47:49.022629 16291 external_mini_cluster.cc:1428] Setting key 251d4e9ef2caf6aa6888eb089e55c2ea
I20260810 06:47:49.024399 31868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e25b60e80564643a1cf6144d69927b1" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42845 } }
I20260810 06:47:49.024542 31868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:49.024566 31868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e25b60e80564643a1cf6144d69927b1, State: Initialized, Role: FOLLOWER
I20260810 06:47:49.024715 31868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e25b60e80564643a1cf6144d69927b1" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42845 } }
I20260810 06:47:49.024849 31868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:47:49.024876 31868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:47:49.024946 31868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:49.025596 31868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e25b60e80564643a1cf6144d69927b1" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42845 } }
I20260810 06:47:49.025686 31868 leader_election.cc:302] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e25b60e80564643a1cf6144d69927b1; no voters:
I20260810 06:47:49.025880 31868 leader_election.cc:288] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:47:49.025910 31873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:47:49.026026 31873 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [term 1 LEADER]: Becoming Leader. State: Replica: 6e25b60e80564643a1cf6144d69927b1, State: Running, Role: LEADER
I20260810 06:47:49.026163 31868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:47:49.026175 31873 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e25b60e80564643a1cf6144d69927b1" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42845 } }
I20260810 06:47:49.026607 31873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e25b60e80564643a1cf6144d69927b1. Latest consensus state: current_term: 1 leader_uuid: "6e25b60e80564643a1cf6144d69927b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e25b60e80564643a1cf6144d69927b1" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42845 } } }
I20260810 06:47:49.026738 31873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:49.026938 31874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e25b60e80564643a1cf6144d69927b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e25b60e80564643a1cf6144d69927b1" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42845 } } }
I20260810 06:47:49.027027 31874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:49.027208 31880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:47:49.027734 31880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:47:49.029635 31880 catalog_manager.cc:1347] Generated new cluster ID: 7242f4dddc1b4dcbb857599dce0b2b6a
I20260810 06:47:49.029690 31880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:47:49 dist-test-slave-vxj2 krb5kdc[31776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:49.035521 31872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:49.023166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.15.232.254:42845)
0810 06:47:49.023386 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:49.023396 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:49.023427 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:47:49.023980 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:49.023988 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:49.024038 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:49.024195 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:49.024983 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:49.026550 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:49.027726 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:49.028488 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:49.028600 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:49.031758 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:49.031762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:49.031762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:49.031774 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:49.033754 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:49.034154 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:49.034157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:49.034158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:49.034257 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:49.034502 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:49.034503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:49.034503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:49.034785 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:49.034944 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:49.035047 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 06:47:49.035175 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
I20260810 06:47:49.036778 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:47:49.038543 31880 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:47:49.038992 31880 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:47:49.047889 31880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e25b60e80564643a1cf6144d69927b1: Generated new TSK 0
I20260810 06:47:49.048139 31880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:47:49.091173 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 31789
2026-08-10T06:47:49Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1495 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:47:49 dist-test-slave-vxj2 krb5kdc[31901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:47:49 dist-test-slave-vxj2 krb5kdc[31901](info): set up 2 sockets
Aug 10 06:47:49 dist-test-slave-vxj2 krb5kdc[31901](info): commencing operation
krb5kdc: starting...
W20260810 06:47:50.240998 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:47:50 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:47:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:47:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:47:50.368614 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:42187
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:42187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:50.452677 31915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:50.452943 31915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:50.452981 31915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:50.454212 31915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:47:50.454285 31915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:50.454309 31915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:50.454329 31915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:47:50.454360 31915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:47:50.455856 31915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:42187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:42187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:50.456143 31915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:50.456387 31915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:50.461442 31924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:50.461455 31923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:50.461438 31927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:50.461745 31915 server_base.cc:1034] running on GCE node
I20260810 06:47:50.462000 31915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:50.462248 31915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:50.463402 31915 hybrid_clock.cc:648] HybridClock initialized: now 1786344470463386 us; error 30 us; skew 500 ppm
Aug 10 06:47:50 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344470, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:50.466005 31915 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:47:50.466475 31915 webserver.cc:460] Webserver started at http://127.15.232.254:37877/ using document root <none> and password file <none>
I20260810 06:47:50.466732 31915 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:50.466778 31915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:50.466909 31915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:47:50.468021 31915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "458afe3edbdc4f65807b997ee4d604b8"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "f11e83a8c85c2b3839b196e4be0987fc"
server_key_iv: "aea56c23ea11e113351ac0cf6b5bf008"
server_key_version: "encryptionkey@0"
I20260810 06:47:50.468390 31915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "458afe3edbdc4f65807b997ee4d604b8"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "f11e83a8c85c2b3839b196e4be0987fc"
server_key_iv: "aea56c23ea11e113351ac0cf6b5bf008"
server_key_version: "encryptionkey@0"
I20260810 06:47:50.469849 31915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:47:50.470654 31935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:50.470937 31915 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:50.471056 31915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "458afe3edbdc4f65807b997ee4d604b8"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "f11e83a8c85c2b3839b196e4be0987fc"
server_key_iv: "aea56c23ea11e113351ac0cf6b5bf008"
server_key_version: "encryptionkey@0"
I20260810 06:47:50.471136 31915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:50.491669 31915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:50.493944 31915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:50.494136 31915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:50.498390 31915 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:42187
I20260810 06:47:50.498435 31995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:42187 every 8 connection(s)
I20260810 06:47:50.498848 31915 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:47:50.499856 31996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:50.502388 31996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Bootstrap starting.
I20260810 06:47:50.502998 31996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:50.503322 31996 log.cc:824] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:50.504046 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 31915
I20260810 06:47:50.504175 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:47:50.504211 31996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: No bootstrap required, opened a new log
I20260810 06:47:50.504393 16291 external_mini_cluster.cc:1428] Setting key db34a982e2760112139bbcce9423add6
I20260810 06:47:50.507041 31996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } }
I20260810 06:47:50.507184 31996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:50.507215 31996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 458afe3edbdc4f65807b997ee4d604b8, State: Initialized, Role: FOLLOWER
I20260810 06:47:50.507359 31996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } }
I20260810 06:47:50.507447 31996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:47:50.507495 31996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:47:50.507546 31996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:50.508188 31996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } }
I20260810 06:47:50.508306 31996 leader_election.cc:302] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 458afe3edbdc4f65807b997ee4d604b8; no voters:
I20260810 06:47:50.508483 31996 leader_election.cc:288] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:47:50.508533 32001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:47:50.508728 32001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 1 LEADER]: Becoming Leader. State: Replica: 458afe3edbdc4f65807b997ee4d604b8, State: Running, Role: LEADER
I20260810 06:47:50.508752 31996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:47:50.510260 32001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } }
I20260810 06:47:50.510859 32002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 458afe3edbdc4f65807b997ee4d604b8. Latest consensus state: current_term: 1 leader_uuid: "458afe3edbdc4f65807b997ee4d604b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } } }
I20260810 06:47:50.510911 32003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "458afe3edbdc4f65807b997ee4d604b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } } }
I20260810 06:47:50.511080 32002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:50.511083 32003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: This master's current role is: LEADER
W20260810 06:47:50.511296 32017 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:47:50.511375 32017 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:47:50.511505 32019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:47:50.511683 32019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:47:50 dist-test-slave-vxj2 krb5kdc[31901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:50.513389 32019 catalog_manager.cc:1347] Generated new cluster ID: f4d55da7faff4853b1bf9d6c282d6d27
I20260810 06:47:50.513451 32019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:47:50.516745 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:50.504951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47180 (local address 127.15.232.254:42187)
0810 06:47:50.505137 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:50.505145 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:50.505222 (+ 77us) server_negotiation.cc:407] Connection header received
0810 06:47:50.505746 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:50.505755 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:50.505812 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:50.505965 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:50.506827 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:50.508204 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:50.509376 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:50.510112 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:50.510230 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:50.513244 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:50.513249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:50.513249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:50.513261 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:50.515046 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:50.515442 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:50.515446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:50.515446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:50.515539 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:50.515757 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:50.515758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:50.515759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:50.516001 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:50.516175 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:50.516259 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 06:47:50.516415 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260810 06:47:50.532680 32019 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:47:50.533515 32019 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:47:50.541600 32019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Generated new TSK 0
I20260810 06:47:50.541852 32019 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:47:50.578207 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:42187
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:50.672490 32024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:50.672715 32024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:50.672741 32024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:50.674029 32024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:50.674077 32024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:50.674105 32024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:47:50.675655 32024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:42187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:50.675866 32024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:50.676069 32024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:50.681299 32032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:50.681282 32024 server_base.cc:1034] running on GCE node
W20260810 06:47:50.681257 32030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:50.681257 32029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:50.681777 32024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:50.682044 32024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:50.683225 32024 hybrid_clock.cc:648] HybridClock initialized: now 1786344470683210 us; error 49 us; skew 500 ppm
Aug 10 06:47:50 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344470, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:50.685992 32024 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:47:50.686527 32024 webserver.cc:460] Webserver started at http://127.15.232.193:39983/ using document root <none> and password file <none>
I20260810 06:47:50.686775 32024 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:50.686846 32024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:50.687000 32024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:47:50.688154 32024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "cc35392c1970450587c602d7bf716f80"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "ba6b383e50fdc62ef5fa695693fadd25"
server_key_iv: "78d99b641714a69f1158760870dccd17"
server_key_version: "encryptionkey@0"
I20260810 06:47:50.688539 32024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "cc35392c1970450587c602d7bf716f80"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "ba6b383e50fdc62ef5fa695693fadd25"
server_key_iv: "78d99b641714a69f1158760870dccd17"
server_key_version: "encryptionkey@0"
I20260810 06:47:50.690013 32024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:47:50.690770 32042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:50.690949 32024 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:47:50.691057 32024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "cc35392c1970450587c602d7bf716f80"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "ba6b383e50fdc62ef5fa695693fadd25"
server_key_iv: "78d99b641714a69f1158760870dccd17"
server_key_version: "encryptionkey@0"
I20260810 06:47:50.691151 32024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:50.702468 32024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:50.704912 32024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:50.705088 32024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:50.705406 32024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:50.705744 32024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:50.705806 32024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:50.705849 32024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:50.705878 32024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:50.712523 32024 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:44599
I20260810 06:47:50.712574 32173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:44599 every 8 connection(s)
I20260810 06:47:50.712942 32024 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:47:50.714346 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 32024
I20260810 06:47:50.714481 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:47:50.714687 16291 external_mini_cluster.cc:1428] Setting key 904112147ad7ec04dfd0437cb9d0f70f
Aug 10 06:47:50 dist-test-slave-vxj2 krb5kdc[31901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344470, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:50.723906 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:50.713714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:38839 (local address 127.15.232.254:42187)
0810 06:47:50.713806 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:50.713809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:50.714049 (+ 240us) server_negotiation.cc:407] Connection header received
0810 06:47:50.714709 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:50.714710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:50.714724 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:50.714777 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:50.715988 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:50.716516 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:50.717680 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:50.717894 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:50.717971 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:50.720583 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:50.720585 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:50.720585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:50.720595 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:50.722387 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:50.722808 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:50.722810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:50.722810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:50.722879 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:50.723141 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:50.723142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:50.723142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:50.723298 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:50.723385 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:50.723702 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 06:47:50.723800 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:47:50.724004 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:50.713670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:38839)
0810 06:47:50.713867 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:50.713955 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:50.714594 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:50.714865 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:50.714870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:50.715668 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:50.715884 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:50.715890 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:50.716620 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:50.716621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:50.717578 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:50.717581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:50.717979 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:50.717980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:50.718131 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:50.718674 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:50.718685 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:50.720448 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:50.722477 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:50.722485 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:50.722487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:50.722722 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:50.722954 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:50.722955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:50.722955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:50.723070 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:50.723390 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:50.723397 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:50.723553 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 06:47:50.723650 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:47:50.723785 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
I20260810 06:47:50.724604 32174 heartbeater.cc:347] Connected to a master server at 127.15.232.254:42187
I20260810 06:47:50.724721 32174 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:50.724918 32174 heartbeater.cc:511] Master 127.15.232.254:42187 requested a full tablet report, sending...
I20260810 06:47:50.725543 31957 ts_manager.cc:194] Registered new tserver with Master: cc35392c1970450587c602d7bf716f80 (127.15.232.193:44599)
I20260810 06:47:50.726130 31957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:38839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:47:50.744057 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:42187
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:50.827137 32181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:50.827399 32181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:50.827438 32181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:50.828574 32181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:50.828650 32181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:50.828697 32181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:47:50.830207 32181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:42187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:50.830499 32181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:50.830739 32181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:47:50.836239 32181 server_base.cc:1034] running on GCE node
W20260810 06:47:50.836190 32190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:50.836215 32194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:50.836215 32191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:50.836678 32181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:50.836956 32181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:50.838109 32181 hybrid_clock.cc:648] HybridClock initialized: now 1786344470838105 us; error 36 us; skew 500 ppm
Aug 10 06:47:50 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344470, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:50.840649 32181 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:47:50.841274 32181 webserver.cc:460] Webserver started at http://127.15.232.194:35693/ using document root <none> and password file <none>
I20260810 06:47:50.841528 32181 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:50.841599 32181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:50.841751 32181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:47:50.842864 32181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "9a88d0e0c8000ea1bd61ccff48a22cb7"
server_key_iv: "2e22558251dfa56ebbf9381f2d1e742c"
server_key_version: "encryptionkey@0"
I20260810 06:47:50.843246 32181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "9a88d0e0c8000ea1bd61ccff48a22cb7"
server_key_iv: "2e22558251dfa56ebbf9381f2d1e742c"
server_key_version: "encryptionkey@0"
I20260810 06:47:50.844609 32181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:47:50.845458 32202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:50.845628 32181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:47:50.845739 32181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "9a88d0e0c8000ea1bd61ccff48a22cb7"
server_key_iv: "2e22558251dfa56ebbf9381f2d1e742c"
server_key_version: "encryptionkey@0"
I20260810 06:47:50.845821 32181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:50.874495 32181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:50.876516 32181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:50.876708 32181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:50.876986 32181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:50.877388 32181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:50.877444 32181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:50.877489 32181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:50.877521 32181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:50.883869 32181 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:33721
I20260810 06:47:50.883985 32333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:33721 every 8 connection(s)
I20260810 06:47:50.884303 32181 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:47:50.889645 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 32181
I20260810 06:47:50.889762 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:47:50.889976 16291 external_mini_cluster.cc:1428] Setting key b0a2facae22a248b974be6d56288069d
Aug 10 06:47:50 dist-test-slave-vxj2 krb5kdc[31901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344470, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:50.894577 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:50.885107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:59873 (local address 127.15.232.254:42187)
0810 06:47:50.885230 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:50.885232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:50.885448 (+ 216us) server_negotiation.cc:407] Connection header received
0810 06:47:50.885980 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:50.885981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:50.885992 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:50.886063 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:50.887136 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:50.887650 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:50.888610 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:50.888789 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:50.888840 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:50.891485 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:50.891487 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:50.891487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:50.891495 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:50.892951 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:50.893480 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:50.893481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:50.893482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:50.893545 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:50.893857 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:50.893858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:50.893858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:50.893977 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:50.894050 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:50.894399 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 06:47:50.894484 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 06:47:50.894838 32338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:50.885091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:59873)
0810 06:47:50.885307 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:50.885352 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:50.885877 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:50.886123 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:50.886126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:50.886790 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:50.887024 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:50.887029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:50.887737 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:50.887738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:50.888532 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:50.888534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:50.888864 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:50.888865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:50.888970 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:50.889536 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:50.889548 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:50.891387 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:50.893055 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:50.893063 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:50.893065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:50.893385 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:50.893628 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:50.893629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:50.893630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:50.893762 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:50.894053 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:50.894061 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:50.894206 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 06:47:50.894311 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 06:47:50.894489 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":141,"threads_started":1}
I20260810 06:47:50.895546 32335 heartbeater.cc:347] Connected to a master server at 127.15.232.254:42187
I20260810 06:47:50.895665 32335 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:50.895855 32335 heartbeater.cc:511] Master 127.15.232.254:42187 requested a full tablet report, sending...
I20260810 06:47:50.896286 31957 ts_manager.cc:194] Registered new tserver with Master: 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721)
I20260810 06:47:50.896811 31957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:59873
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:47:50.920432 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:42187
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:51.004418 32343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:51.004648 32343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:51.004673 32343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:51.006027 32343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:51.006075 32343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:51.006104 32343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:47:51.007619 32343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:42187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:51.007879 32343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:51.008123 32343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:51.013574 32348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:51.013616 32350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:51.013618 32353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:51.013803 32343 server_base.cc:1034] running on GCE node
I20260810 06:47:51.014021 32343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:51.014281 32343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:51.015450 32343 hybrid_clock.cc:648] HybridClock initialized: now 1786344471015423 us; error 56 us; skew 500 ppm
Aug 10 06:47:51 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344471, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:51.018004 32343 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:47:51.018522 32343 webserver.cc:460] Webserver started at http://127.15.232.195:38327/ using document root <none> and password file <none>
I20260810 06:47:51.018779 32343 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:51.018849 32343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:51.019006 32343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:47:51.020107 32343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "30e4bfb9260c4b4c8c6c6c2482be8865"
format_stamp: "Formatted at 2026-08-10 06:47:51 on dist-test-slave-vxj2"
server_key: "35f3eb0e2de44e074b70260533773296"
server_key_iv: "bdf765ed782b1641ff455f244ebd023d"
server_key_version: "encryptionkey@0"
I20260810 06:47:51.020498 32343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "30e4bfb9260c4b4c8c6c6c2482be8865"
format_stamp: "Formatted at 2026-08-10 06:47:51 on dist-test-slave-vxj2"
server_key: "35f3eb0e2de44e074b70260533773296"
server_key_iv: "bdf765ed782b1641ff455f244ebd023d"
server_key_version: "encryptionkey@0"
I20260810 06:47:51.021974 32343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:47:51.022749 32361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:51.022924 32343 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:47:51.023025 32343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "30e4bfb9260c4b4c8c6c6c2482be8865"
format_stamp: "Formatted at 2026-08-10 06:47:51 on dist-test-slave-vxj2"
server_key: "35f3eb0e2de44e074b70260533773296"
server_key_iv: "bdf765ed782b1641ff455f244ebd023d"
server_key_version: "encryptionkey@0"
I20260810 06:47:51.023106 32343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:51.053366 32343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:51.055670 32343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:51.055861 32343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:51.056145 32343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:51.056493 32343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:51.056552 32343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:51.056594 32343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:51.056623 32343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:51.063002 32343 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:38151
I20260810 06:47:51.063050 32486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:38151 every 8 connection(s)
I20260810 06:47:51.063387 32343 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:47:51.066313 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 32343
I20260810 06:47:51.066396 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:47:51.066601 16291 external_mini_cluster.cc:1428] Setting key 1fd9c12407ce642d615a0c2f195d18bc
Aug 10 06:47:51 dist-test-slave-vxj2 krb5kdc[31901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344471, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:51.073889 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.064213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:44189 (local address 127.15.232.254:42187)
0810 06:47:51.064291 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.064293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.064485 (+ 192us) server_negotiation.cc:407] Connection header received
0810 06:47:51.064969 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.064970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.064981 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.065041 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:51.066090 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.066731 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.068092 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.068287 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.068346 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.070747 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:51.070749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:51.070749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:51.070765 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:51.072307 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.072800 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.072801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.072802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.072869 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.073145 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.073146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.073147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.073299 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:51.073389 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.073697 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.073784 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:47:51.074129 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.064171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:44189)
0810 06:47:51.064332 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.064377 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:51.064881 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:51.065106 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:51.065110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:51.065783 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:51.065997 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.066003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.066844 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.066845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.068006 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.068010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.068367 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.068368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.068492 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.069008 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:51.069017 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:51.070633 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:51.072442 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:51.072453 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:51.072455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:51.072715 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:51.072943 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:51.072944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:51.072944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:51.073076 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:51.073371 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:51.073379 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:51.073541 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 06:47:51.073663 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 06:47:51.073868 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260810 06:47:51.074723 32487 heartbeater.cc:347] Connected to a master server at 127.15.232.254:42187
I20260810 06:47:51.074812 32487 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:51.075023 32487 heartbeater.cc:511] Master 127.15.232.254:42187 requested a full tablet report, sending...
I20260810 06:47:51.075408 31957 ts_manager.cc:194] Registered new tserver with Master: 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151)
I20260810 06:47:51.075810 31957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:44189
I20260810 06:47:51.078495 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:47:51.086655 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.079879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.15.232.254:42187)
0810 06:47:51.079964 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.079966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.080084 (+ 118us) server_negotiation.cc:407] Connection header received
0810 06:47:51.080163 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.080165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.080175 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.080246 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:51.080856 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.081560 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.082381 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.082575 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.082663 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.083952 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:51.083954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:51.083954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:51.083963 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:51.085312 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.085713 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.085715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.085715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.085782 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.086004 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.086005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.086005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.086179 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:51.086293 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.086442 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.086547 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:47:51.088480 31957 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:47:51.090200 31957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:47:51.097401 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.093090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46776 (local address 127.15.232.195:38151)
0810 06:47:51.093338 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.093342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.093529 (+ 187us) server_negotiation.cc:407] Connection header received
0810 06:47:51.093572 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.093574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.093591 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.093837 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:51.093900 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.095018 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.096157 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.096917 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.097004 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.097067 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.097270 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.097316 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":135,"threads_started":1}
I20260810 06:47:51.097429 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.092638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33721 (local address 127.0.0.1:58220)
0810 06:47:51.093069 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.093120 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:51.093262 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:51.093618 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:51.093621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:51.093623 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:51.093701 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.093703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.095042 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.095043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.096136 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.096139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.097030 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.097031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.097135 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.097143 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:47:51.097272 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 06:47:51.097307 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":69,"threads_started":1}
I20260810 06:47:51.097429 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.092638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38151 (local address 127.0.0.1:46776)
0810 06:47:51.093389 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.093400 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:51.093465 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:51.093720 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:51.093721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:51.093723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:51.093767 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.093769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.095102 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.095103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.096052 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.096055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.097028 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.097029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.097126 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.097143 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 06:47:51.097237 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 06:47:51.097301 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":13,"spinlock_wait_cycles":896,"thread_start_us":120,"threads_started":1}
I20260810 06:47:51.098053 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.092820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58220 (local address 127.15.232.194:33721)
0810 06:47:51.093028 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.093034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.093234 (+ 200us) server_negotiation.cc:407] Connection header received
0810 06:47:51.093399 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.093403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.093424 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.093514 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:51.093786 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.094933 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.096214 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.096917 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.097005 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.097067 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.097576 (+ 509us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.097620 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":125,"threads_started":1}
I20260810 06:47:51.098357 32502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.092638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:44599 (local address 127.0.0.1:55462)
0810 06:47:51.093262 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.093273 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:51.093324 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:51.093687 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:51.093688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:51.093690 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:51.093763 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.093765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.094738 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.094739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.096137 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.096139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.098086 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.098087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.098164 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.098168 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:47:51.098226 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 06:47:51.098271 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":50,"thread_start_us":97,"threads_started":1}
I20260810 06:47:51.098402 32412 tablet_service.cc:1467] Processing CreateTablet for tablet fc3844d8163646a8a2c20deb1ec03dcb (DEFAULT_TABLE table=test-table [id=46decadd1fc04f80b69f7c48dc9161da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:51.098650 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.092837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55462 (local address 127.15.232.193:44599)
0810 06:47:51.093056 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.093061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.093325 (+ 264us) server_negotiation.cc:407] Connection header received
0810 06:47:51.093498 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.093503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.093524 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.093599 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:51.093850 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.094635 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.097111 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.097926 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.097994 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.098050 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.098517 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.098561 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":145,"threads_started":1}
I20260810 06:47:51.098862 32412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc3844d8163646a8a2c20deb1ec03dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:51.098819 32260 tablet_service.cc:1467] Processing CreateTablet for tablet fc3844d8163646a8a2c20deb1ec03dcb (DEFAULT_TABLE table=test-table [id=46decadd1fc04f80b69f7c48dc9161da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:51.099187 32260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc3844d8163646a8a2c20deb1ec03dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:51.102458 32509 tablet_bootstrap.cc:492] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865: Bootstrap starting.
I20260810 06:47:51.102449 32507 tablet_bootstrap.cc:492] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc: Bootstrap starting.
I20260810 06:47:51.103211 32509 tablet_bootstrap.cc:654] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:51.103211 32507 tablet_bootstrap.cc:654] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:51.103169 32102 tablet_service.cc:1467] Processing CreateTablet for tablet fc3844d8163646a8a2c20deb1ec03dcb (DEFAULT_TABLE table=test-table [id=46decadd1fc04f80b69f7c48dc9161da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:51.103499 32102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc3844d8163646a8a2c20deb1ec03dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:51.103837 32507 log.cc:824] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:51.103837 32509 log.cc:824] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:51.104705 32507 tablet_bootstrap.cc:492] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc: No bootstrap required, opened a new log
I20260810 06:47:51.104764 32507 ts_tablet_manager.cc:1397] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:47:51.105556 32509 tablet_bootstrap.cc:492] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865: No bootstrap required, opened a new log
I20260810 06:47:51.105680 32509 ts_tablet_manager.cc:1397] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:47:51.106352 32507 raft_consensus.cc:348] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } }
I20260810 06:47:51.106539 32507 raft_consensus.cc:374] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:51.106588 32507 raft_consensus.cc:729] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c3cdf8179e4ba29bc934f17f7d0bdc, State: Initialized, Role: FOLLOWER
I20260810 06:47:51.106700 32507 consensus_queue.cc:260] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } }
I20260810 06:47:51.107017 32507 ts_tablet_manager.cc:1428] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:47:51.106994 32512 tablet_bootstrap.cc:492] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Bootstrap starting.
I20260810 06:47:51.107172 32335 heartbeater.cc:503] Master 127.15.232.254:42187 was elected leader, sending a full tablet report...
I20260810 06:47:51.107769 32512 tablet_bootstrap.cc:654] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:51.108259 32509 raft_consensus.cc:348] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } }
I20260810 06:47:51.108376 32512 log.cc:824] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:51.108444 32509 raft_consensus.cc:374] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:51.108494 32509 raft_consensus.cc:729] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30e4bfb9260c4b4c8c6c6c2482be8865, State: Initialized, Role: FOLLOWER
I20260810 06:47:51.108621 32509 consensus_queue.cc:260] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } }
I20260810 06:47:51.108955 32487 heartbeater.cc:503] Master 127.15.232.254:42187 was elected leader, sending a full tablet report...
I20260810 06:47:51.109290 32512 tablet_bootstrap.cc:492] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: No bootstrap required, opened a new log
I20260810 06:47:51.109287 32509 ts_tablet_manager.cc:1428] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:47:51.109381 32512 ts_tablet_manager.cc:1397] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:47:51.111097 32512 raft_consensus.cc:348] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } }
I20260810 06:47:51.111239 32512 raft_consensus.cc:374] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:51.111263 32512 raft_consensus.cc:729] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc35392c1970450587c602d7bf716f80, State: Initialized, Role: FOLLOWER
I20260810 06:47:51.111343 32512 consensus_queue.cc:260] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } }
I20260810 06:47:51.111565 32512 ts_tablet_manager.cc:1428] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:47:51.111691 32174 heartbeater.cc:503] Master 127.15.232.254:42187 was elected leader, sending a full tablet report...
W20260810 06:47:51.134933 32337 tablet.cc:2369] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:47:51.213665 32175 tablet.cc:2369] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:47:51.276804 32516 raft_consensus.cc:482] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:47:51.276947 32516 raft_consensus.cc:504] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } }
I20260810 06:47:51.277459 32516 leader_election.cc:288] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721), cc35392c1970450587c602d7bf716f80 (127.15.232.193:44599)
I20260810 06:47:51.281633 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.277740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:41773 (local address 127.15.232.193:44599)
0810 06:47:51.277788 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.277790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.278026 (+ 236us) server_negotiation.cc:407] Connection header received
0810 06:47:51.278082 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.278083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.278090 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.278168 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:51.278310 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.279042 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.280552 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.281123 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.281232 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.281241 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.281484 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.281526 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:47:51.281600 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.277714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:35933 (local address 127.15.232.194:33721)
0810 06:47:51.277775 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.277777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.277814 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:47:51.277845 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.277846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.277857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.277931 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:51.278143 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.278908 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.280552 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.281136 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.281233 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.281241 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.281484 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.281525 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 06:47:51.281620 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.277639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33721 (local address 127.15.232.195:35933)
0810 06:47:51.277703 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.277716 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:51.277764 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:51.277965 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:51.277967 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:51.277969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:51.278053 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.278055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.279007 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.279008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.280448 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.280451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.281283 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.281284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.281382 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.281384 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:47:51.281459 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:47:51.281515 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":16}
I20260810 06:47:51.281628 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.277672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:44599 (local address 127.15.232.195:41773)
0810 06:47:51.277942 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.277954 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:51.278019 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:51.278190 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:51.278191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:51.278193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:51.278242 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.278243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.279120 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.279121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.280448 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.280451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.281280 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.281281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.281377 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.281382 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:47:51.281465 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:47:51.281514 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":6,"thread_start_us":127,"threads_started":1}
I20260810 06:47:51.282029 32125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3844d8163646a8a2c20deb1ec03dcb" candidate_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc35392c1970450587c602d7bf716f80" is_pre_election: true
I20260810 06:47:51.282089 32286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3844d8163646a8a2c20deb1ec03dcb" candidate_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" is_pre_election: true
I20260810 06:47:51.282187 32125 raft_consensus.cc:2454] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30e4bfb9260c4b4c8c6c6c2482be8865 in term 0.
I20260810 06:47:51.282306 32286 raft_consensus.cc:2454] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30e4bfb9260c4b4c8c6c6c2482be8865 in term 0.
I20260810 06:47:51.282470 32363 leader_election.cc:302] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30e4bfb9260c4b4c8c6c6c2482be8865, cc35392c1970450587c602d7bf716f80; no voters:
I20260810 06:47:51.282642 32516 raft_consensus.cc:2781] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:47:51.282739 32516 raft_consensus.cc:482] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:47:51.282769 32516 raft_consensus.cc:3037] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:51.283542 32516 raft_consensus.cc:504] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } }
I20260810 06:47:51.283706 32516 leader_election.cc:288] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [CANDIDATE]: Term 1 election: Requested vote from peers 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721), cc35392c1970450587c602d7bf716f80 (127.15.232.193:44599)
I20260810 06:47:51.283850 32286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3844d8163646a8a2c20deb1ec03dcb" candidate_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc"
I20260810 06:47:51.283934 32286 raft_consensus.cc:3037] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:51.283896 32125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3844d8163646a8a2c20deb1ec03dcb" candidate_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc35392c1970450587c602d7bf716f80"
I20260810 06:47:51.284021 32125 raft_consensus.cc:3037] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:51.284613 32286 raft_consensus.cc:2454] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30e4bfb9260c4b4c8c6c6c2482be8865 in term 1.
I20260810 06:47:51.284818 32366 leader_election.cc:302] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c3cdf8179e4ba29bc934f17f7d0bdc, 30e4bfb9260c4b4c8c6c6c2482be8865; no voters:
I20260810 06:47:51.284907 32125 raft_consensus.cc:2454] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30e4bfb9260c4b4c8c6c6c2482be8865 in term 1.
I20260810 06:47:51.284925 32516 raft_consensus.cc:2781] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:47:51.285127 32516 raft_consensus.cc:686] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 1 LEADER]: Becoming Leader. State: Replica: 30e4bfb9260c4b4c8c6c6c2482be8865, State: Running, Role: LEADER
I20260810 06:47:51.285261 32516 consensus_queue.cc:237] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } }
I20260810 06:47:51.286231 31956 catalog_manager.cc:5495] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195). New cstate: current_term: 1 leader_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } health_report { overall_health: HEALTHY } } }
W20260810 06:47:51.313968 32489 tablet.cc:2369] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:47:51.324330 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.322025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46786 (local address 127.15.232.195:38151)
0810 06:47:51.322093 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.322095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.322120 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:47:51.322158 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.322159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.322176 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.322230 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:47:51.322374 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.322842 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.323588 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.323775 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.323828 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.323980 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:47:51.324086 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:47:51.324156 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.324174 (+ 18us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.324192 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:47:51.326386 32125 raft_consensus.cc:1261] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Refusing update from remote peer 30e4bfb9260c4b4c8c6c6c2482be8865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:47:51.326403 32286 raft_consensus.cc:1261] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 1 FOLLOWER]: Refusing update from remote peer 30e4bfb9260c4b4c8c6c6c2482be8865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:47:51.326819 32520 consensus_queue.cc:1035] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:47:51.326979 32516 consensus_queue.cc:1035] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:47:51.329736 32523 mvcc.cc:205] Tried to move back new op lower bound from 7316866954549710848 to 7316866954385281024. Current Snapshot: MvccSnapshot[applied={T|T < 7316866954549710848}]
I20260810 06:47:51.329819 32524 mvcc.cc:205] Tried to move back new op lower bound from 7316866954549710848 to 7316866954385281024. Current Snapshot: MvccSnapshot[applied={T|T < 7316866954549710848}]
I20260810 06:47:51.330560 32525 mvcc.cc:205] Tried to move back new op lower bound from 7316866954549710848 to 7316866954385281024. Current Snapshot: MvccSnapshot[applied={T|T < 7316866954549710848}]
I20260810 06:47:51.332552 31956 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:47:51.332670 31956 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:47:51.333513 31956 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Sending DeleteTablet for 3 replicas of tablet fc3844d8163646a8a2c20deb1ec03dcb
I20260810 06:47:51.333875 32412 tablet_service.cc:1514] Processing DeleteTablet for tablet fc3844d8163646a8a2c20deb1ec03dcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:51 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:46776
I20260810 06:47:51.334004 32102 tablet_service.cc:1514] Processing DeleteTablet for tablet fc3844d8163646a8a2c20deb1ec03dcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:51 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:55462
I20260810 06:47:51.334255 32534 tablet_replica.cc:330] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865: stopping tablet replica
I20260810 06:47:51.334301 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 32024
I20260810 06:47:51.334394 32534 raft_consensus.cc:2229] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:47:51.334533 32534 raft_consensus.cc:2258] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:47:51.335968 32534 ts_tablet_manager.cc:1905] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:51.337937 32534 ts_tablet_manager.cc:1918] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:47:51.338023 32534 log.cc:1196] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/fc3844d8163646a8a2c20deb1ec03dcb
I20260810 06:47:51.338331 32534 ts_tablet_manager.cc:1939] T fc3844d8163646a8a2c20deb1ec03dcb P 30e4bfb9260c4b4c8c6c6c2482be8865: Deleting consensus metadata
I20260810 06:47:51.333911 32260 tablet_service.cc:1514] Processing DeleteTablet for tablet fc3844d8163646a8a2c20deb1ec03dcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:51 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:58220
I20260810 06:47:51.339131 31937 catalog_manager.cc:4841] TS 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151): tablet fc3844d8163646a8a2c20deb1ec03dcb (table test-table [id=46decadd1fc04f80b69f7c48dc9161da]) successfully deleted
I20260810 06:47:51.339164 32536 tablet_replica.cc:330] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc: stopping tablet replica
I20260810 06:47:51.339368 32536 raft_consensus.cc:2229] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:47:51.339514 32536 raft_consensus.cc:2258] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:47:51.341775 32536 ts_tablet_manager.cc:1905] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:51.343567 32536 ts_tablet_manager.cc:1918] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:47:51.343662 32536 log.cc:1196] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/fc3844d8163646a8a2c20deb1ec03dcb
I20260810 06:47:51.344149 32536 ts_tablet_manager.cc:1939] T fc3844d8163646a8a2c20deb1ec03dcb P 15c3cdf8179e4ba29bc934f17f7d0bdc: Deleting consensus metadata
W20260810 06:47:51.344204 31938 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:44599 (error 108)
I20260810 06:47:51.344408 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 32181
I20260810 06:47:51.351799 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.351619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:44599 (local address 127.0.0.1:55470)
0810 06:47:51.351695 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.351746 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:44599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:47:51.351981 31938 catalog_manager.cc:4584] TS cc35392c1970450587c602d7bf716f80 (127.15.232.193:44599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fc3844d8163646a8a2c20deb1ec03dcb: Network error: Client connection negotiation failed: client connection to 127.15.232.193:44599: connect: Connection refused (error 111)
I20260810 06:47:51.358059 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.357898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33721 (local address 127.0.0.1:58228)
0810 06:47:51.357973 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.358008 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.194:33721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 06:47:51.358084 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 32343
I20260810 06:47:51.366245 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 31915
I20260810 06:47:51.376042 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:42187
--webserver_interface=127.15.232.254
--webserver_port=37877
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:42187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:51.464506 32538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:51.464766 32538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:51.464802 32538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:51.466087 32538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:47:51.466136 32538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:51.466150 32538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:51.466163 32538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:47:51.466176 32538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:47:51.467689 32538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:42187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:42187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=37877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:51.467912 32538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:51.468096 32538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:51.478037 32544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:51.478039 32546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:51.478183 32549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:51.478474 32538 server_base.cc:1034] running on GCE node
I20260810 06:47:51.478722 32538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:51.479007 32538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:51.480149 32538 hybrid_clock.cc:648] HybridClock initialized: now 1786344471480137 us; error 28 us; skew 500 ppm
Aug 10 06:47:51 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344471, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:51.482828 32538 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:47:51.483259 32538 webserver.cc:460] Webserver started at http://127.15.232.254:37877/ using document root <none> and password file <none>
I20260810 06:47:51.483491 32538 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:51.483547 32538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:51.484941 32538 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:47:51.485698 32559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:51.485992 32538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:47:51.486105 32538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "458afe3edbdc4f65807b997ee4d604b8"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "f11e83a8c85c2b3839b196e4be0987fc"
server_key_iv: "aea56c23ea11e113351ac0cf6b5bf008"
server_key_version: "encryptionkey@0"
I20260810 06:47:51.486655 32538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:51.513059 32538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:51.515241 32538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:51.515400 32538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:51.519661 32538 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:42187
I20260810 06:47:51.519783 32620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:42187 every 8 connection(s)
I20260810 06:47:51.520146 32538 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:47:51.522094 32621 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:47:51.522444 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 32538
I20260810 06:47:51.522740 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:44599
--local_ip_for_outbound_sockets=127.15.232.193
--tserver_master_addrs=127.15.232.254:42187
--webserver_port=39983
--webserver_interface=127.15.232.193
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:47:51.523013 32621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Bootstrap starting.
I20260810 06:47:51.532364 32621 log.cc:824] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:51.534933 32621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:47:51.535306 32621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Bootstrap complete.
I20260810 06:47:51.537802 32621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } }
I20260810 06:47:51.538084 32621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 458afe3edbdc4f65807b997ee4d604b8, State: Initialized, Role: FOLLOWER
I20260810 06:47:51.538301 32621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } }
I20260810 06:47:51.538395 32621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:47:51.538465 32621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:47:51.538527 32621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:47:51.539677 32621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } }
I20260810 06:47:51.539819 32621 leader_election.cc:302] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 458afe3edbdc4f65807b997ee4d604b8; no voters:
I20260810 06:47:51.540071 32621 leader_election.cc:288] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:47:51.540117 32626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:47:51.540247 32626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 2 LEADER]: Becoming Leader. State: Replica: 458afe3edbdc4f65807b997ee4d604b8, State: Running, Role: LEADER
I20260810 06:47:51.540362 32626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } }
I20260810 06:47:51.540402 32621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:47:51.540688 32628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 458afe3edbdc4f65807b997ee4d604b8. Latest consensus state: current_term: 2 leader_uuid: "458afe3edbdc4f65807b997ee4d604b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } } }
I20260810 06:47:51.540910 32628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:51.540848 32627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "458afe3edbdc4f65807b997ee4d604b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } } }
I20260810 06:47:51.541210 32627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:51.541275 32635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:47:51.542205 32635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:47:51.542318 32635 catalog_manager.cc:1259] Loaded cluster ID: f4d55da7faff4853b1bf9d6c282d6d27
I20260810 06:47:51.542347 32635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:47:51.542866 32635 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 06:47:51.547343 32635 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 06:47:51.549997 32635 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:47:51.550242 32635 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Loaded TSK: 0
I20260810 06:47:51.550477 32635 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:47:51.610683 32623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:51.610941 32623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:51.610981 32623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:51.612185 32623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:51.612262 32623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:51.612309 32623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:47:51.613868 32623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:44599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=39983
--tserver_master_addrs=127.15.232.254:42187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:51.614159 32623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:51.614405 32623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:51.619731 32656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:51.619731 32653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:51.620121 32623 server_base.cc:1034] running on GCE node
W20260810 06:47:51.619868 32658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:51.620427 32623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:51.620707 32623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:51.621892 32623 hybrid_clock.cc:648] HybridClock initialized: now 1786344471621866 us; error 50 us; skew 500 ppm
Aug 10 06:47:51 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344471, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:51.624938 32623 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:47:51.625483 32623 webserver.cc:460] Webserver started at http://127.15.232.193:39983/ using document root <none> and password file <none>
I20260810 06:47:51.625726 32623 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:51.625813 32623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:51.627408 32623 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:47:51.628235 32667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:51.628456 32623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:51.628564 32623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "cc35392c1970450587c602d7bf716f80"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "ba6b383e50fdc62ef5fa695693fadd25"
server_key_iv: "78d99b641714a69f1158760870dccd17"
server_key_version: "encryptionkey@0"
I20260810 06:47:51.628950 32623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:51.655917 32623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:51.658350 32623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:51.658565 32623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:51.658872 32623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:51.659374 32674 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 06:47:51.661407 32623 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 06:47:51.661489 32623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 06:47:51.661535 32623 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 06:47:51.662186 32623 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 06:47:51.662245 32623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 06:47:51.662329 32674 tablet_bootstrap.cc:492] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Bootstrap starting.
I20260810 06:47:51.669910 32623 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:44599
I20260810 06:47:51.670331 32623 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:47:51.674877 32674 log.cc:824] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:51.675262 342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:44599 every 8 connection(s)
Aug 10 06:47:51 dist-test-slave-vxj2 krb5kdc[31901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344471, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:51.678884 32674 tablet_bootstrap.cc:492] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:47:51.679028 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 32623
I20260810 06:47:51.679253 32674 tablet_bootstrap.cc:492] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Bootstrap complete.
I20260810 06:47:51.679821 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:33721
--local_ip_for_outbound_sockets=127.15.232.194
--tserver_master_addrs=127.15.232.254:42187
--webserver_port=35693
--webserver_interface=127.15.232.194
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:47:51.680598 32674 ts_tablet_manager.cc:1397] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.010s
I20260810 06:47:51.682109 32674 raft_consensus.cc:348] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } }
I20260810 06:47:51.682488 32674 raft_consensus.cc:729] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc35392c1970450587c602d7bf716f80, State: Initialized, Role: FOLLOWER
I20260810 06:47:51.682644 32674 consensus_queue.cc:260] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } }
I20260810 06:47:51.682948 32674 ts_tablet_manager.cc:1428] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:47:51.683560 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.671225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:40697 (local address 127.15.232.254:42187)
0810 06:47:51.671421 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.671431 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.671668 (+ 237us) server_negotiation.cc:407] Connection header received
0810 06:47:51.672329 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.672334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.672377 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.672528 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:51.673225 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.674364 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.675693 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.676114 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.676242 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.678962 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:51.678968 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:51.678968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:51.678983 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:51.681379 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.681958 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.681962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.681962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.682107 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.682380 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.682381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.682381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.682777 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:51.682861 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.683221 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.683352 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":75,"threads_started":1}
I20260810 06:47:51.683583 345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.671145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:40697)
0810 06:47:51.671516 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.671550 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:51.672084 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:51.672558 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:51.672561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:51.672871 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:51.673067 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.673073 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.674511 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.674513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.675599 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.675603 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.676213 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.676215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.676361 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.676944 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:51.676960 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:51.678740 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:51.681480 (+ 2740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:51.681488 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:51.681490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:51.681831 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:51.682194 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:51.682195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:51.682195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:51.682294 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:51.682851 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:51.682859 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:51.683019 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 06:47:51.683157 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 06:47:51.683348 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":107,"threads_started":1}
I20260810 06:47:51.684643 343 heartbeater.cc:347] Connected to a master server at 127.15.232.254:42187
I20260810 06:47:51.684746 343 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:51.684942 343 heartbeater.cc:511] Master 127.15.232.254:42187 requested a full tablet report, sending...
I20260810 06:47:51.685814 32580 ts_manager.cc:194] Registered new tserver with Master: cc35392c1970450587c602d7bf716f80 (127.15.232.193:44599)
I20260810 06:47:51.687078 32580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:40697
I20260810 06:47:51.688207 343 heartbeater.cc:503] Master 127.15.232.254:42187 was elected leader, sending a full tablet report...
Aug 10 06:47:51 dist-test-slave-vxj2 krb5kdc[31901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344471, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.193@KRBTEST.COM
I20260810 06:47:51.697846 353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.686756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55474 (local address 127.15.232.193:44599)
0810 06:47:51.687644 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.687648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.687659 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:47:51.687695 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.687697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.687722 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.687788 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:51.688364 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.689687 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.691220 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.691660 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.691767 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.694657 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:51.694669 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:51.694670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:51.694681 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:51.696391 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.696856 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.696859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.696859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.696926 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.697137 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.697138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.697138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.697379 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:51.697455 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.697657 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.697773 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":93,"threads_started":1}
I20260810 06:47:51.697870 352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.686486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:44599 (local address 127.0.0.1:55474)
0810 06:47:51.687396 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.687431 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:51.687505 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:51.687820 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:51.687825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:51.688168 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:51.688234 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.688236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.689832 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.689834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.691103 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.691108 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.691768 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.691770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.691863 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.692471 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:51.692478 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:51.694534 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:51.696478 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:51.696485 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:51.696488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:51.696764 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:51.697000 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:51.697001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:51.697001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:51.697072 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:51.697453 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:51.697460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:51.697520 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 06:47:51.697614 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 06:47:51.697759 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"thread_start_us":82,"threads_started":1}
I20260810 06:47:51.698359 32729 tablet_service.cc:1514] Processing DeleteTablet for tablet fc3844d8163646a8a2c20deb1ec03dcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:51 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:55474
I20260810 06:47:51.699028 354 tablet_replica.cc:330] stopping tablet replica
I20260810 06:47:51.699164 354 raft_consensus.cc:2229] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:47:51.699288 354 raft_consensus.cc:2258] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:47:51.699637 354 ts_tablet_manager.cc:1905] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:51.702039 354 ts_tablet_manager.cc:1918] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:47:51.702131 354 log.cc:1196] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal/wals/fc3844d8163646a8a2c20deb1ec03dcb
I20260810 06:47:51.702405 354 ts_tablet_manager.cc:1939] T fc3844d8163646a8a2c20deb1ec03dcb P cc35392c1970450587c602d7bf716f80: Deleting consensus metadata
I20260810 06:47:51.702994 32562 catalog_manager.cc:4841] TS cc35392c1970450587c602d7bf716f80 (127.15.232.193:44599): tablet fc3844d8163646a8a2c20deb1ec03dcb (table test-table [id=46decadd1fc04f80b69f7c48dc9161da]) successfully deleted
W20260810 06:47:51.771445 349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:51.771698 349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:51.771739 349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:51.772907 349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:51.772979 349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:51.773020 349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:47:51.774530 349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:33721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=35693
--tserver_master_addrs=127.15.232.254:42187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:51.774808 349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:51.775084 349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:51.780416 360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:51.780405 361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:51.780406 363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:51.780664 349 server_base.cc:1034] running on GCE node
I20260810 06:47:51.780954 349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:51.781261 349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:51.782434 349 hybrid_clock.cc:648] HybridClock initialized: now 1786344471782424 us; error 39 us; skew 500 ppm
Aug 10 06:47:51 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344471, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:51.784961 349 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:47:51.785514 349 webserver.cc:460] Webserver started at http://127.15.232.194:35693/ using document root <none> and password file <none>
I20260810 06:47:51.785761 349 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:51.785856 349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:51.787320 349 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:47:51.788100 372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:51.788416 349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:47:51.788645 349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "9a88d0e0c8000ea1bd61ccff48a22cb7"
server_key_iv: "2e22558251dfa56ebbf9381f2d1e742c"
server_key_version: "encryptionkey@0"
I20260810 06:47:51.789079 349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:51.797875 349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:51.799729 349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:51.799911 349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:51.800173 349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:51.800508 349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:51.800563 349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:51.800608 349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:51.800633 349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:51.806831 349 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:33721
I20260810 06:47:51.806876 509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:33721 every 8 connection(s)
I20260810 06:47:51.807258 349 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:47:51.808265 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 349
I20260810 06:47:51.808622 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:38151
--local_ip_for_outbound_sockets=127.15.232.195
--tserver_master_addrs=127.15.232.254:42187
--webserver_port=38327
--webserver_interface=127.15.232.195
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:47:51 dist-test-slave-vxj2 krb5kdc[31901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344471, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:51.818861 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.808005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:35813)
0810 06:47:51.808199 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.808257 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:51.809063 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:51.809412 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:51.809416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:51.810301 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:51.810549 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.810555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.811586 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.811587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.812440 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.812443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.813021 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.813023 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.813210 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.813829 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:51.813838 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:51.815285 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:51.817395 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:51.817403 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:51.817405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:51.817647 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:51.817882 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:51.817882 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:51.817883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:51.817999 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:51.818319 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:51.818325 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:51.818456 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 06:47:51.818547 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 06:47:51.818682 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260810 06:47:51.818859 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.808060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:35813 (local address 127.15.232.254:42187)
0810 06:47:51.808152 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.808155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.808397 (+ 242us) server_negotiation.cc:407] Connection header received
0810 06:47:51.809255 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.809257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.809272 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.809314 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:51.810674 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.811476 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.812538 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.812896 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.813002 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.815387 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:51.815389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:51.815389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:51.815399 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:51.817312 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.817730 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.817731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.817732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.817802 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.818070 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.818072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.818072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.818243 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:51.818327 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.818602 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.818715 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:47:51.819501 510 heartbeater.cc:347] Connected to a master server at 127.15.232.254:42187
I20260810 06:47:51.819628 510 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:51.819837 510 heartbeater.cc:511] Master 127.15.232.254:42187 requested a full tablet report, sending...
I20260810 06:47:51.820313 32580 ts_manager.cc:194] Registered new tserver with Master: 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721)
I20260810 06:47:51.820716 32580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:35813
W20260810 06:47:51.896276 513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:51.896551 513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:51.896596 513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:51.897940 513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:51.898018 513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:51.898066 513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:47:51.899637 513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:38151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=38327
--tserver_master_addrs=127.15.232.254:42187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:51.899948 513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:51.900207 513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:51.905725 522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:51.905728 519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:51.905728 520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:51.906183 513 server_base.cc:1034] running on GCE node
I20260810 06:47:51.906412 513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:51.906713 513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:51.907881 513 hybrid_clock.cc:648] HybridClock initialized: now 1786344471907865 us; error 43 us; skew 500 ppm
Aug 10 06:47:51 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344471, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:51.910696 513 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:47:51.911208 513 webserver.cc:460] Webserver started at http://127.15.232.195:38327/ using document root <none> and password file <none>
I20260810 06:47:51.911427 513 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:51.911494 513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:51.912957 513 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:51.913657 529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:51.913846 513 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:47:51.913944 513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "30e4bfb9260c4b4c8c6c6c2482be8865"
format_stamp: "Formatted at 2026-08-10 06:47:51 on dist-test-slave-vxj2"
server_key: "35f3eb0e2de44e074b70260533773296"
server_key_iv: "bdf765ed782b1641ff455f244ebd023d"
server_key_version: "encryptionkey@0"
I20260810 06:47:51.914281 513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:51.923256 513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:51.925318 513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:51.925458 513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:51.925657 513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:51.925974 513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:51.926009 513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:51.926100 513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:51.926168 513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:51.932428 513 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:38151
I20260810 06:47:51.932525 665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:38151 every 8 connection(s)
I20260810 06:47:51.932915 513 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:47:51.935026 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 513
Aug 10 06:47:51 dist-test-slave-vxj2 krb5kdc[31901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344471, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:47:51.943516 670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.935465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.15.232.254:42187)
0810 06:47:51.935614 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.935616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.935667 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:47:51.935738 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.935739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.935747 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.935803 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:51.936508 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.937244 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.938288 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.938515 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.938583 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.940113 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:51.940114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:51.940114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:51.940121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:51.942307 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.942657 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.942659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.942659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.942721 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.942914 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.942915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.942915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.943065 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:51.943195 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.943304 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.943371 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":103,"threads_started":1}
I20260810 06:47:51.944777 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.933787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:38965 (local address 127.15.232.254:42187)
0810 06:47:51.933864 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.933866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.934095 (+ 229us) server_negotiation.cc:407] Connection header received
0810 06:47:51.934588 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.934589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.934599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.934665 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:51.936296 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.937063 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.938537 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.938751 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.938798 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.941439 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:51.941440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:51.941441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:51.941448 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:51.942966 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.943443 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.943444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.943444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.943499 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:51.943966 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:51.943967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:51.943967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:51.944126 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:51.944192 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.944647 (+ 455us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.944717 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:47:51.945030 668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.933757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:38965)
0810 06:47:51.933947 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.933995 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:51.934501 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:51.934721 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:51.934724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:51.935854 (+ 1130us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:51.936186 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.936194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.937210 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.937211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.938459 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.938461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.938842 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.938843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.938989 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.939404 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:51.939414 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:51.941341 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:51.943040 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:51.943046 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:51.943048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:51.943368 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:51.943723 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:51.943724 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:51.943724 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:51.943864 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:51.944365 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:51.944371 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:51.944528 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 06:47:51.944619 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 06:47:51.944749 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
I20260810 06:47:51.945586 666 heartbeater.cc:347] Connected to a master server at 127.15.232.254:42187
I20260810 06:47:51.945703 666 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:51.945897 666 heartbeater.cc:511] Master 127.15.232.254:42187 requested a full tablet report, sending...
I20260810 06:47:51.946309 32580 ts_manager.cc:194] Registered new tserver with Master: 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151)
I20260810 06:47:51.946692 32580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:38965
I20260810 06:47:51.955567 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:47:51.958636 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.955973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47206 (local address 127.15.232.254:42187)
0810 06:47:51.956047 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.956050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.956149 (+ 99us) server_negotiation.cc:407] Connection header received
0810 06:47:51.956243 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.956244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.956288 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.956371 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:47:51.956522 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.957034 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.957847 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.958044 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.958122 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.958305 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:47:51.958407 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:47:51.958502 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.958525 (+ 23us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.958545 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:47:51.958910 32580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:47:51.960487 32580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:47:51.962917 32729 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd316ff44b843aea809bcf599141406 (DEFAULT_TABLE table=test-table [id=f0764651bc1548d08a67e9ad70769c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:51.963151 32729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd316ff44b843aea809bcf599141406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:51.964502 32674 tablet_bootstrap.cc:492] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80: Bootstrap starting.
I20260810 06:47:51.964918 32674 tablet_bootstrap.cc:654] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:51.965566 32674 tablet_bootstrap.cc:492] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80: No bootstrap required, opened a new log
I20260810 06:47:51.965619 32674 ts_tablet_manager.cc:1397] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260810 06:47:51.965969 32674 raft_consensus.cc:348] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:51.966066 32674 raft_consensus.cc:374] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:51.966118 32674 raft_consensus.cc:729] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc35392c1970450587c602d7bf716f80, State: Initialized, Role: FOLLOWER
I20260810 06:47:51.966173 32674 consensus_queue.cc:260] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:51.966262 32674 ts_tablet_manager.cc:1428] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 06:47:51.967183 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.962627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38151 (local address 127.0.0.1:46796)
0810 06:47:51.962809 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.962819 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:51.962863 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:51.963259 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:51.963265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:51.963268 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:51.963334 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.963336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.964673 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.964674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.965928 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.965932 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.966913 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.966913 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.966979 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.967001 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 06:47:51.967053 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:47:51.967102 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":39,"thread_start_us":115,"threads_started":1}
I20260810 06:47:51.967300 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.962696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58230 (local address 127.15.232.194:33721)
0810 06:47:51.962954 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.962959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.962968 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:47:51.963040 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.963042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.963060 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.963186 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:51.963360 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.964536 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.965924 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.966758 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.966855 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.966920 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.967151 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.967201 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":123,"threads_started":1}
I20260810 06:47:51.967299 352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.962534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33721 (local address 127.0.0.1:58230)
0810 06:47:51.962568 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:51.962583 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:51.962627 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:51.963203 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:51.963205 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:51.963207 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:51.963259 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.963260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.964673 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.964674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.965821 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:51.965825 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.966978 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.966978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:51.967050 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.967053 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:47:51.967115 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 06:47:51.967166 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":47}
I20260810 06:47:51.967525 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:51.962880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46796 (local address 127.15.232.195:38151)
0810 06:47:51.963091 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:51.963095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:51.963104 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:47:51.963141 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:51.963143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:51.963158 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:51.963413 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:51.963520 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.964568 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.966040 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:51.966831 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:51.966904 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:51.966958 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:51.967381 (+ 423us) server_negotiation.cc:299] Negotiation successful
0810 06:47:51.967426 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":109,"threads_started":1}
I20260810 06:47:51.968236 587 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd316ff44b843aea809bcf599141406 (DEFAULT_TABLE table=test-table [id=f0764651bc1548d08a67e9ad70769c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:51.968236 429 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd316ff44b843aea809bcf599141406 (DEFAULT_TABLE table=test-table [id=f0764651bc1548d08a67e9ad70769c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:51.968611 429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd316ff44b843aea809bcf599141406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:51.968611 587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd316ff44b843aea809bcf599141406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:51.972442 677 tablet_bootstrap.cc:492] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865: Bootstrap starting.
I20260810 06:47:51.973095 677 tablet_bootstrap.cc:654] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:51.973428 676 tablet_bootstrap.cc:492] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc: Bootstrap starting.
I20260810 06:47:51.973964 677 log.cc:824] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:51.974135 676 tablet_bootstrap.cc:654] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:51.974712 676 log.cc:824] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:51.974903 677 tablet_bootstrap.cc:492] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865: No bootstrap required, opened a new log
I20260810 06:47:51.974985 677 ts_tablet_manager.cc:1397] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:47:51.975596 676 tablet_bootstrap.cc:492] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc: No bootstrap required, opened a new log
I20260810 06:47:51.975673 676 ts_tablet_manager.cc:1397] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:47:51.977121 677 raft_consensus.cc:348] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:51.977344 677 raft_consensus.cc:374] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:51.977384 677 raft_consensus.cc:729] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30e4bfb9260c4b4c8c6c6c2482be8865, State: Initialized, Role: FOLLOWER
I20260810 06:47:51.977524 677 consensus_queue.cc:260] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:51.977850 666 heartbeater.cc:503] Master 127.15.232.254:42187 was elected leader, sending a full tablet report...
I20260810 06:47:51.977865 677 ts_tablet_manager.cc:1428] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260810 06:47:51.977830 676 raft_consensus.cc:348] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:51.977993 676 raft_consensus.cc:374] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:51.978044 676 raft_consensus.cc:729] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c3cdf8179e4ba29bc934f17f7d0bdc, State: Initialized, Role: FOLLOWER
I20260810 06:47:51.978183 676 consensus_queue.cc:260] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:51.978477 676 ts_tablet_manager.cc:1428] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:47:51.978536 510 heartbeater.cc:503] Master 127.15.232.254:42187 was elected leader, sending a full tablet report...
W20260810 06:47:52.057843 511 tablet.cc:2369] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:47:52.136268 351 raft_consensus.cc:482] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:47:52.136456 351 raft_consensus.cc:504] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:52.136847 351 leader_election.cc:288] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721), 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151)
I20260810 06:47:52.140161 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.137137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:38597 (local address 127.15.232.194:33721)
0810 06:47:52.137215 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:52.137217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:52.137392 (+ 175us) server_negotiation.cc:407] Connection header received
0810 06:47:52.137522 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:52.137523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:52.137535 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:52.137606 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:52.137782 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:52.138254 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:52.139335 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:52.139737 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:52.139833 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:52.139854 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:52.140038 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 06:47:52.140081 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:47:52.140187 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.137080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33721 (local address 127.15.232.193:38597)
0810 06:47:52.137280 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:52.137292 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:52.137424 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:52.137610 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:52.137611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:52.137613 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:52.137705 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:52.137707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:52.138354 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:52.138355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:52.139224 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:52.139227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:52.139848 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:52.139849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:52.139932 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:52.139957 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 06:47:52.140031 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 06:47:52.140077 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":42,"thread_start_us":104,"threads_started":1}
I20260810 06:47:52.140259 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.137212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:44483 (local address 127.15.232.195:38151)
0810 06:47:52.137295 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:52.137298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:52.137310 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:47:52.137441 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:52.137443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:52.137454 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:52.137511 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:52.137696 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:52.138262 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:52.139395 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:52.139835 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:52.139930 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:52.139939 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:52.140125 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 06:47:52.140173 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:47:52.140276 345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.137078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38151 (local address 127.15.232.193:44483)
0810 06:47:52.137201 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:52.137214 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:52.137291 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:52.137515 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:52.137516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:52.137518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:52.137610 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:52.137612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:52.138361 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:52.138362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:52.139282 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:52.139285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:52.139928 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:52.139929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:52.140018 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:52.140021 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:47:52.140108 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:47:52.140154 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":58}
I20260810 06:47:52.140939 449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd316ff44b843aea809bcf599141406" candidate_uuid: "cc35392c1970450587c602d7bf716f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" is_pre_election: true
I20260810 06:47:52.141057 611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd316ff44b843aea809bcf599141406" candidate_uuid: "cc35392c1970450587c602d7bf716f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" is_pre_election: true
I20260810 06:47:52.141208 449 raft_consensus.cc:2454] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc35392c1970450587c602d7bf716f80 in term 0.
I20260810 06:47:52.141209 611 raft_consensus.cc:2454] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc35392c1970450587c602d7bf716f80 in term 0.
I20260810 06:47:52.141494 32671 leader_election.cc:302] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c3cdf8179e4ba29bc934f17f7d0bdc, cc35392c1970450587c602d7bf716f80; no voters:
I20260810 06:47:52.141678 351 raft_consensus.cc:2781] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:47:52.141752 351 raft_consensus.cc:482] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:47:52.141824 351 raft_consensus.cc:3037] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:52.142699 351 raft_consensus.cc:504] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:52.142884 351 leader_election.cc:288] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 1 election: Requested vote from peers 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721), 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151)
I20260810 06:47:52.143059 449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd316ff44b843aea809bcf599141406" candidate_uuid: "cc35392c1970450587c602d7bf716f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc"
I20260810 06:47:52.143148 449 raft_consensus.cc:3037] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:52.143138 611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd316ff44b843aea809bcf599141406" candidate_uuid: "cc35392c1970450587c602d7bf716f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865"
I20260810 06:47:52.143236 611 raft_consensus.cc:3037] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:52.143986 449 raft_consensus.cc:2454] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc35392c1970450587c602d7bf716f80 in term 1.
I20260810 06:47:52.144104 611 raft_consensus.cc:2454] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc35392c1970450587c602d7bf716f80 in term 1.
I20260810 06:47:52.144176 32671 leader_election.cc:302] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c3cdf8179e4ba29bc934f17f7d0bdc, cc35392c1970450587c602d7bf716f80; no voters:
I20260810 06:47:52.144296 351 raft_consensus.cc:2781] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:47:52.144484 351 raft_consensus.cc:686] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 LEADER]: Becoming Leader. State: Replica: cc35392c1970450587c602d7bf716f80, State: Running, Role: LEADER
I20260810 06:47:52.144574 351 consensus_queue.cc:237] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:52.145491 32579 catalog_manager.cc:5495] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc35392c1970450587c602d7bf716f80 (127.15.232.193). New cstate: current_term: 1 leader_uuid: "cc35392c1970450587c602d7bf716f80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } health_report { overall_health: HEALTHY } } }
I20260810 06:47:52.151895 353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.149473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55486 (local address 127.15.232.193:44599)
0810 06:47:52.149520 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:52.149521 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:52.149529 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:47:52.149614 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:52.149615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:52.149631 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:52.149690 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:47:52.149813 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:52.150373 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:52.151128 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:52.151332 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:52.151393 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:52.151539 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:47:52.151671 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:47:52.151729 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:52.151765 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 06:47:52.151786 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:47:52.153676 611 raft_consensus.cc:1261] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 1 FOLLOWER]: Refusing update from remote peer cc35392c1970450587c602d7bf716f80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:47:52.153695 449 raft_consensus.cc:1261] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 1 FOLLOWER]: Refusing update from remote peer cc35392c1970450587c602d7bf716f80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:47:52.154062 351 consensus_queue.cc:1035] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:47:52.154241 685 consensus_queue.cc:1035] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:47:52.156859 688 mvcc.cc:205] Tried to move back new op lower bound from 7316866957938737152 to 7316866957905027072. Current Snapshot: MvccSnapshot[applied={T|T < 7316866957938737152}]
I20260810 06:47:52.159473 32579 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:47:52.159591 32579 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:47:52.160351 32579 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Sending DeleteTablet for 3 replicas of tablet 6fd316ff44b843aea809bcf599141406
I20260810 06:47:52.160679 587 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd316ff44b843aea809bcf599141406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:52 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:46796
I20260810 06:47:52.160706 429 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd316ff44b843aea809bcf599141406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:52 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:58230
I20260810 06:47:52.161048 32729 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd316ff44b843aea809bcf599141406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:52 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:55474
I20260810 06:47:52.161098 700 tablet_replica.cc:330] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc: stopping tablet replica
I20260810 06:47:52.161098 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 32623
I20260810 06:47:52.161145 354 tablet_replica.cc:330] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80: stopping tablet replica
I20260810 06:47:52.161213 354 raft_consensus.cc:2229] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:47:52.161288 354 raft_consensus.cc:2258] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:47:52.162940 700 raft_consensus.cc:2229] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:47:52.163066 700 raft_consensus.cc:2258] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:47:52.166754 701 tablet_replica.cc:330] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865: stopping tablet replica
I20260810 06:47:52.167018 701 raft_consensus.cc:2229] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:47:52.167188 701 raft_consensus.cc:2258] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:47:52.167209 700 ts_tablet_manager.cc:1905] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:52.167659 701 ts_tablet_manager.cc:1905] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:52.169776 700 ts_tablet_manager.cc:1918] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:47:52.169901 700 log.cc:1196] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/6fd316ff44b843aea809bcf599141406
I20260810 06:47:52.170033 701 ts_tablet_manager.cc:1918] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:47:52.170140 701 log.cc:1196] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/6fd316ff44b843aea809bcf599141406
I20260810 06:47:52.170210 700 ts_tablet_manager.cc:1939] T 6fd316ff44b843aea809bcf599141406 P 15c3cdf8179e4ba29bc934f17f7d0bdc: Deleting consensus metadata
I20260810 06:47:52.170480 701 ts_tablet_manager.cc:1939] T 6fd316ff44b843aea809bcf599141406 P 30e4bfb9260c4b4c8c6c6c2482be8865: Deleting consensus metadata
I20260810 06:47:52.170799 32564 catalog_manager.cc:4841] TS 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721): tablet 6fd316ff44b843aea809bcf599141406 (table test-table [id=f0764651bc1548d08a67e9ad70769c4f]) successfully deleted
I20260810 06:47:52.170986 32560 catalog_manager.cc:4841] TS 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151): tablet 6fd316ff44b843aea809bcf599141406 (table test-table [id=f0764651bc1548d08a67e9ad70769c4f]) successfully deleted
W20260810 06:47:52.172801 32562 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:44599 (error 108)
I20260810 06:47:52.173415 352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.173260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:44599 (local address 127.0.0.1:55490)
0810 06:47:52.173323 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:52.173377 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:44599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 06:47:52.173527 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 349
W20260810 06:47:52.173568 32562 catalog_manager.cc:4584] TS cc35392c1970450587c602d7bf716f80 (127.15.232.193:44599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6fd316ff44b843aea809bcf599141406: Network error: Client connection negotiation failed: client connection to 127.15.232.193:44599: connect: Connection refused (error 111)
I20260810 06:47:52.182657 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 513
I20260810 06:47:52.191781 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 32538
I20260810 06:47:52.200286 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:42187
--webserver_interface=127.15.232.254
--webserver_port=37877
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:42187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:52.286684 703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:52.286950 703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:52.286990 703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:52.288228 703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:47:52.288308 703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:52.288332 703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:52.288353 703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:47:52.288393 703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:47:52.289995 703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:42187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:42187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=37877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:52.290302 703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:52.290571 703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:52.295722 709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:52.295704 710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:52.295993 703 server_base.cc:1034] running on GCE node
W20260810 06:47:52.296051 712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:52.296329 703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:52.296639 703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:52.297811 703 hybrid_clock.cc:648] HybridClock initialized: now 1786344472297804 us; error 38 us; skew 500 ppm
Aug 10 06:47:52 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344472, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:52.300475 703 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:47:52.300917 703 webserver.cc:460] Webserver started at http://127.15.232.254:37877/ using document root <none> and password file <none>
I20260810 06:47:52.301119 703 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:52.301237 703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:52.303233 703 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:47:52.304121 720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:52.304306 703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:47:52.304416 703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "458afe3edbdc4f65807b997ee4d604b8"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "f11e83a8c85c2b3839b196e4be0987fc"
server_key_iv: "aea56c23ea11e113351ac0cf6b5bf008"
server_key_version: "encryptionkey@0"
I20260810 06:47:52.304768 703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:52.331171 703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:52.333662 703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:52.333793 703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:52.338069 703 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:42187
I20260810 06:47:52.338136 784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:42187 every 8 connection(s)
I20260810 06:47:52.338544 703 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:47:52.340188 787 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:47:52.340904 787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Bootstrap starting.
I20260810 06:47:52.346661 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 703
I20260810 06:47:52.346989 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:44599
--local_ip_for_outbound_sockets=127.15.232.193
--tserver_master_addrs=127.15.232.254:42187
--webserver_port=39983
--webserver_interface=127.15.232.193
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:47:52.349648 787 log.cc:824] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:52.351987 787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:47:52.352305 787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8: Bootstrap complete.
I20260810 06:47:52.354691 787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } }
I20260810 06:47:52.354962 787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 458afe3edbdc4f65807b997ee4d604b8, State: Initialized, Role: FOLLOWER
I20260810 06:47:52.355161 787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } }
I20260810 06:47:52.355247 787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:47:52.355302 787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:47:52.355356 787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:47:52.356419 787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } }
I20260810 06:47:52.356562 787 leader_election.cc:302] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 458afe3edbdc4f65807b997ee4d604b8; no voters:
I20260810 06:47:52.356781 787 leader_election.cc:288] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 06:47:52.356832 791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:47:52.356966 791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [term 3 LEADER]: Becoming Leader. State: Replica: 458afe3edbdc4f65807b997ee4d604b8, State: Running, Role: LEADER
I20260810 06:47:52.357084 787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:47:52.357084 791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } }
I20260810 06:47:52.357463 794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "458afe3edbdc4f65807b997ee4d604b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } } }
I20260810 06:47:52.357473 792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 458afe3edbdc4f65807b997ee4d604b8. Latest consensus state: current_term: 3 leader_uuid: "458afe3edbdc4f65807b997ee4d604b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458afe3edbdc4f65807b997ee4d604b8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42187 } } }
I20260810 06:47:52.357769 794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:52.357790 792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 458afe3edbdc4f65807b997ee4d604b8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:52.358266 802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:47:52.359156 802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:47:52.359522 802 catalog_manager.cc:1259] Loaded cluster ID: f4d55da7faff4853b1bf9d6c282d6d27
I20260810 06:47:52.359565 802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 06:47:52.359865 802 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1fa019c9ad google::LogMessage::Fail() at ??:0
@ 0x7f1fa019f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f1fa019c4de google::LogMessage::Flush() at ??:0
@ 0x7f1fa019da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1fa80bd0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f1fa80c68b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1fa206a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1fa205f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1fa2ec36db start_thread at ??:0
@ 0x7f1f9ec9a61f clone at ??:0
W20260810 06:47:52.445536 789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:52.445771 789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:52.445811 789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:52.447027 789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:52.447103 789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:52.447152 789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:47:52.448673 789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:44599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=39983
--tserver_master_addrs=127.15.232.254:42187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:52.448961 789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:52.449256 789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:52.454759 826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:52.454759 827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:52.455075 789 server_base.cc:1034] running on GCE node
W20260810 06:47:52.454767 829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:52.455345 789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:52.455624 789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:52.456794 789 hybrid_clock.cc:648] HybridClock initialized: now 1786344472456782 us; error 33 us; skew 500 ppm
Aug 10 06:47:52 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344472, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:52.459546 789 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:47:52.460044 789 webserver.cc:460] Webserver started at http://127.15.232.193:39983/ using document root <none> and password file <none>
I20260810 06:47:52.460278 789 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:52.460350 789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:52.461939 789 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:47:52.462704 836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:52.462891 789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:47:52.463028 789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "cc35392c1970450587c602d7bf716f80"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "ba6b383e50fdc62ef5fa695693fadd25"
server_key_iv: "78d99b641714a69f1158760870dccd17"
server_key_version: "encryptionkey@0"
I20260810 06:47:52.463392 789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:52.475270 789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:52.477900 789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:52.478086 789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:52.478384 789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:52.478926 844 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 06:47:52.480984 789 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 06:47:52.481055 789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 06:47:52.481122 789 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 06:47:52.481843 789 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 06:47:52.481925 789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 06:47:52.481994 844 tablet_bootstrap.cc:492] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80: Bootstrap starting.
I20260810 06:47:52.489467 789 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:44599
I20260810 06:47:52.489621 965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:44599 every 8 connection(s)
I20260810 06:47:52.489935 789 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:47:52.492802 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 789
I20260810 06:47:52.493103 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:33721
--local_ip_for_outbound_sockets=127.15.232.194
--tserver_master_addrs=127.15.232.254:42187
--webserver_port=35693
--webserver_interface=127.15.232.194
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:47:52.493716 844 log.cc:824] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:52.496790 844 tablet_bootstrap.cc:492] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:47:52.497131 844 tablet_bootstrap.cc:492] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80: Bootstrap complete.
I20260810 06:47:52.498467 844 ts_tablet_manager.cc:1397] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.012s
I20260810 06:47:52.499958 844 raft_consensus.cc:348] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:52.500315 844 raft_consensus.cc:729] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc35392c1970450587c602d7bf716f80, State: Initialized, Role: FOLLOWER
I20260810 06:47:52.500509 844 consensus_queue.cc:260] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:52.500818 844 ts_tablet_manager.cc:1428] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260810 06:47:52.565055 968 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:47:52.490653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:49875)
0810 06:47:52.490826 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:52.490871 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:52.491393 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:52.564842 (+ 73449us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
W20260810 06:47:52.565372 966 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:42187 (0 consecutive failures): Network error: Failed to ping master at 127.15.232.254:42187: Client connection negotiation failed: client connection to 127.15.232.254:42187: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:47:52.565768 968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.565626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:39055)
0810 06:47:52.565691 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:52.565742 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 06:47:52.566263 968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.566142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:56515)
0810 06:47:52.566195 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:52.566229 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 06:47:52.566416 966 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:47:52.580729 970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:52.580987 970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:52.581014 970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:52.582237 970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:52.582284 970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:52.582314 970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:47:52.583774 970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:33721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=35693
--tserver_master_addrs=127.15.232.254:42187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:52.583972 970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:52.584167 970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:52.589568 979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:52.589684 970 server_base.cc:1034] running on GCE node
W20260810 06:47:52.589576 977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:52.589612 981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:52.590044 970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:52.590271 970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:52.591432 970 hybrid_clock.cc:648] HybridClock initialized: now 1786344472591400 us; error 29 us; skew 500 ppm
Aug 10 06:47:52 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344472, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:52.593994 970 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:47:52.594507 970 webserver.cc:460] Webserver started at http://127.15.232.194:35693/ using document root <none> and password file <none>
I20260810 06:47:52.594736 970 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:52.594786 970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:52.596184 970 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:47:52.596803 988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:52.596988 970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:47:52.597106 970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc"
format_stamp: "Formatted at 2026-08-10 06:47:50 on dist-test-slave-vxj2"
server_key: "9a88d0e0c8000ea1bd61ccff48a22cb7"
server_key_iv: "2e22558251dfa56ebbf9381f2d1e742c"
server_key_version: "encryptionkey@0"
I20260810 06:47:52.597551 970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:52.610225 970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:52.612573 970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:52.612823 970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:52.613111 970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:52.613495 970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:52.613569 970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:52.613615 970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:52.613639 970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:52.620087 970 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:33721
I20260810 06:47:52.620137 1123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:33721 every 8 connection(s)
I20260810 06:47:52.620504 970 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:47:52.621719 1126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.621312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:57795)
0810 06:47:52.621481 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:52.621571 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
W20260810 06:47:52.622227 1124 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:42187 (0 consecutive failures): Network error: Failed to ping master at 127.15.232.254:42187: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
I20260810 06:47:52.622581 1126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.622482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:52997)
0810 06:47:52.622533 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:52.622563 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 06:47:52.622923 1126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.622846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:59333)
0810 06:47:52.622887 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:52.622909 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 06:47:52.623051 1124 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:47:52.629106 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 970
I20260810 06:47:52.629451 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:38151
--local_ip_for_outbound_sockets=127.15.232.195
--tserver_master_addrs=127.15.232.254:42187
--webserver_port=38327
--webserver_interface=127.15.232.195
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:52.714039 1127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:52.714269 1127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:52.714298 1127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:52.715567 1127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:52.715621 1127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:52.715687 1127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:47:52.717288 1127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:38151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=38327
--tserver_master_addrs=127.15.232.254:42187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:47:52.717581 1127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:52.717839 1127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:52.723272 1135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:52.723413 1127 server_base.cc:1034] running on GCE node
W20260810 06:47:52.723306 1132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:52.723311 1133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:52.723735 1127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:52.723954 1127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:52.725139 1127 hybrid_clock.cc:648] HybridClock initialized: now 1786344472725113 us; error 42 us; skew 500 ppm
Aug 10 06:47:52 dist-test-slave-vxj2 krb5kdc[31901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344472, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:52.731515 1127 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:47:52.732080 1127 webserver.cc:460] Webserver started at http://127.15.232.195:38327/ using document root <none> and password file <none>
I20260810 06:47:52.732342 1127 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:52.732419 1127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:52.734041 1127 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:47:52.734917 1144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:52.735226 1127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:47:52.735459 1127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "30e4bfb9260c4b4c8c6c6c2482be8865"
format_stamp: "Formatted at 2026-08-10 06:47:51 on dist-test-slave-vxj2"
server_key: "35f3eb0e2de44e074b70260533773296"
server_key_iv: "bdf765ed782b1641ff455f244ebd023d"
server_key_version: "encryptionkey@0"
I20260810 06:47:52.735855 1127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:52.749047 1127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:52.752233 1127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:52.752408 1127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:52.752687 1127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:52.753031 1127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:52.753067 1127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:52.753090 1127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:52.753134 1127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:52.759443 1127 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:38151
I20260810 06:47:52.759524 1272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:38151 every 8 connection(s)
I20260810 06:47:52.759927 1127 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:47:52.761130 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.760703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:36691)
0810 06:47:52.760874 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:52.760982 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
W20260810 06:47:52.761588 1275 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:42187 (0 consecutive failures): Network error: Failed to ping master at 127.15.232.254:42187: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
I20260810 06:47:52.761899 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.761792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:33547)
0810 06:47:52.761843 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:52.761870 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 06:47:52.762174 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:52.762088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:58487)
0810 06:47:52.762133 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:52.762156 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 06:47:52.762288 1275 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:47:52.764926 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 1127
I20260810 06:47:53.567166 1279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:53.566880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:49409)
0810 06:47:53.567027 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:53.567094 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260810 06:47:53.623759 1280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:53.623474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:34293)
0810 06:47:53.623652 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:53.623698 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":65,"threads_started":1}
I20260810 06:47:53.763000 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:53.762708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:49873)
0810 06:47:53.762883 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:53.762934 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260810 06:47:54.298795 1282 raft_consensus.cc:482] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:47:54.299015 1282 raft_consensus.cc:504] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:54.299454 1282 leader_election.cc:288] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721), 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151)
Aug 10 06:47:54 dist-test-slave-vxj2 krb5kdc[31901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344472, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.194@KRBTEST.COM
Aug 10 06:47:54 dist-test-slave-vxj2 krb5kdc[31901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344472, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.195@KRBTEST.COM
I20260810 06:47:54.311079 1288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:54.299835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:56153 (local address 127.15.232.194:33721)
0810 06:47:54.299999 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:54.300010 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:54.300031 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:47:54.300476 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:54.300479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:54.300583 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:54.300734 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:54.301581 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:54.302466 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:54.303622 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:54.304057 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:54.304192 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:54.306876 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:54.306882 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:54.306882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:54.306894 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:54.308911 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:54.309469 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:54.309473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:54.309473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:54.309575 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:54.309915 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:54.309916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:54.309917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:54.310330 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:54.310428 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:54.310784 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 06:47:54.310935 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
I20260810 06:47:54.311053 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:54.299610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:33721 (local address 127.15.232.193:56153)
0810 06:47:54.299744 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:54.299758 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:54.299828 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:54.300777 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:54.300779 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:54.301257 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:54.301457 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:54.301464 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:54.302594 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:54.302595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:54.303495 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:54.303498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:54.304165 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:54.304166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:54.304284 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:54.304964 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:54.304974 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:54.306732 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:54.309026 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:54.309035 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:54.309037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:54.309379 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:54.309648 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:54.309649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:54.309649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:54.309816 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:54.310416 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:54.310425 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:54.310578 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 06:47:54.310691 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 06:47:54.310922 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":91,"thread_start_us":78,"threads_started":1}
I20260810 06:47:54.311138 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:54.299818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:46357 (local address 127.15.232.195:38151)
0810 06:47:54.300045 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:54.300057 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:54.300085 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:47:54.300593 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:54.300597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:54.300654 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:54.300809 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:54.301597 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:54.302545 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:54.303603 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:54.304058 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:54.304204 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:54.307294 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:54.307298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:54.307298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:54.307310 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:54.309123 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:54.309920 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:54.309923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:54.309924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:54.310031 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:54.310262 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:54.310262 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:54.310263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:54.310498 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:54.310589 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:54.310870 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 06:47:54.311021 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":67,"threads_started":1}
I20260810 06:47:54.311327 1284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:54.299627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38151 (local address 127.15.232.193:46357)
0810 06:47:54.299895 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:54.299909 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:54.299970 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:54.300807 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:54.300808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:54.301257 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:54.301458 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:54.301464 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:54.302659 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:54.302660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:54.303496 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:54.303499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:54.304165 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:54.304166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:54.304287 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:54.305011 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:54.305017 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:54.307156 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:54.309311 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:54.309312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:54.309313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:54.309560 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:54.310108 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:54.310109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:54.310109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:54.310181 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:54.310574 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:54.310575 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:54.310623 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:47:54.310711 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 06:47:54.311015 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":168,"thread_start_us":112,"threads_started":1}
W20260810 06:47:54.312338 840 leader_election.cc:341] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721): Not found: Tablet not found: 6fd316ff44b843aea809bcf599141406
W20260810 06:47:54.312448 837 leader_election.cc:341] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151): Not found: Tablet not found: 6fd316ff44b843aea809bcf599141406
I20260810 06:47:54.312493 837 leader_election.cc:302] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc35392c1970450587c602d7bf716f80; no voters: 15c3cdf8179e4ba29bc934f17f7d0bdc, 30e4bfb9260c4b4c8c6c6c2482be8865
I20260810 06:47:54.312642 1282 raft_consensus.cc:2726] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:47:54.567889 1284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:54.567696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:44677)
0810 06:47:54.567806 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:54.567848 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 06:47:54.624646 1289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:54.624384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:38095)
0810 06:47:54.624535 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:54.624596 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260810 06:47:54.763826 1290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:54.763581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:35995)
0810 06:47:54.763713 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:54.763771 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260810 06:47:55.568665 1291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:55.568414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:44397)
0810 06:47:55.568578 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:55.568617 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260810 06:47:55.625648 1292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:55.625397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:48999)
0810 06:47:55.625554 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:55.625601 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260810 06:47:55.764614 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:55.764346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:43541)
0810 06:47:55.764517 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:55.764569 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":72,"threads_started":1}
I20260810 06:47:56.234869 1294 raft_consensus.cc:482] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:47:56.234992 1294 raft_consensus.cc:504] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:56.235237 1294 leader_election.cc:288] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721), 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151)
W20260810 06:47:56.235752 837 leader_election.cc:341] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151): Not found: Tablet not found: 6fd316ff44b843aea809bcf599141406
W20260810 06:47:56.235867 840 leader_election.cc:341] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721): Not found: Tablet not found: 6fd316ff44b843aea809bcf599141406
I20260810 06:47:56.235935 840 leader_election.cc:302] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc35392c1970450587c602d7bf716f80; no voters: 15c3cdf8179e4ba29bc934f17f7d0bdc, 30e4bfb9260c4b4c8c6c6c2482be8865
I20260810 06:47:56.236014 1294 raft_consensus.cc:2726] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:47:56.569550 1295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:56.569290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:36583)
0810 06:47:56.569439 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:56.569484 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260810 06:47:56.626503 1296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:56.626195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:57899)
0810 06:47:56.626344 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:56.626399 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":53,"threads_started":1}
I20260810 06:47:56.765476 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:56.765223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:44769)
0810 06:47:56.765385 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:56.765431 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":64,"threads_started":1}
I20260810 06:47:57.570376 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:57.570128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:45949)
0810 06:47:57.570277 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:57.570326 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260810 06:47:57.627434 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:57.627154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:43107)
0810 06:47:57.627331 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:57.627383 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260810 06:47:57.766356 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:57.766129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:39541)
0810 06:47:57.766258 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:57.766306 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260810 06:47:58.571297 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:58.571026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:34933)
0810 06:47:58.571199 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:58.571246 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":73,"threads_started":1}
I20260810 06:47:58.628335 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:58.628063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:46007)
0810 06:47:58.628218 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:58.628288 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
I20260810 06:47:58.767242 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:58.766982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:34883)
0810 06:47:58.767139 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:58.767191 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260810 06:47:59.572153 1305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:59.571928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:53709)
0810 06:47:59.572059 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:59.572104 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260810 06:47:59.584076 1306 raft_consensus.cc:482] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:47:59.584187 1306 raft_consensus.cc:504] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:47:59.584546 1306 leader_election.cc:288] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721), 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151)
W20260810 06:47:59.585063 840 leader_election.cc:341] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721): Not found: Tablet not found: 6fd316ff44b843aea809bcf599141406
W20260810 06:47:59.585152 837 leader_election.cc:341] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151): Not found: Tablet not found: 6fd316ff44b843aea809bcf599141406
I20260810 06:47:59.585246 837 leader_election.cc:302] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc35392c1970450587c602d7bf716f80; no voters: 15c3cdf8179e4ba29bc934f17f7d0bdc, 30e4bfb9260c4b4c8c6c6c2482be8865
I20260810 06:47:59.585346 1306 raft_consensus.cc:2726] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:47:59.629312 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:59.628960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:44177)
0810 06:47:59.629200 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:59.629251 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":111,"threads_started":1}
I20260810 06:47:59.768131 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:59.767881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:53149)
0810 06:47:59.768036 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:59.768083 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260810 06:48:00.573026 1309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:00.572759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:52195)
0810 06:48:00.572929 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:00.572975 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":69,"threads_started":1}
I20260810 06:48:00.630227 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:00.629940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:50751)
0810 06:48:00.630124 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:00.630176 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":91,"threads_started":1}
I20260810 06:48:00.769114 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:00.768828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:51791)
0810 06:48:00.769013 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:00.769063 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":96,"threads_started":1}
I20260810 06:48:01.573844 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:01.573577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:44193)
0810 06:48:01.573731 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:01.573791 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":70,"threads_started":1}
I20260810 06:48:01.631181 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:01.630896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:44703)
0810 06:48:01.631074 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:01.631127 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":102,"threads_started":1}
I20260810 06:48:01.770129 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:01.769826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:37329)
0810 06:48:01.770027 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:01.770078 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":108,"threads_started":1}
I20260810 06:48:02.574707 1315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:02.574455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:60387)
0810 06:48:02.574609 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:02.574655 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":68,"threads_started":1}
I20260810 06:48:02.632115 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:02.631831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:43695)
0810 06:48:02.632016 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:02.632064 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
I20260810 06:48:02.771056 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:02.770755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:41021)
0810 06:48:02.770958 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:02.771006 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":134,"threads_started":1}
I20260810 06:48:03.575501 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:03.575260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:55083)
0810 06:48:03.575410 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:03.575453 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
I20260810 06:48:03.633013 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:03.632725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:53889)
0810 06:48:03.632907 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:03.632961 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":86,"threads_started":1}
I20260810 06:48:03.771929 1320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:03.771634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:48695)
0810 06:48:03.771828 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:03.771879 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":90,"threads_started":1}
I20260810 06:48:04.396584 1321 raft_consensus.cc:482] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:04.396680 1321 raft_consensus.cc:504] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c3cdf8179e4ba29bc934f17f7d0bdc" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 33721 } } peers { permanent_uuid: "30e4bfb9260c4b4c8c6c6c2482be8865" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38151 } } peers { permanent_uuid: "cc35392c1970450587c602d7bf716f80" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 44599 } }
I20260810 06:48:04.396976 1321 leader_election.cc:288] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721), 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151)
W20260810 06:48:04.397286 837 leader_election.cc:341] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30e4bfb9260c4b4c8c6c6c2482be8865 (127.15.232.195:38151): Not found: Tablet not found: 6fd316ff44b843aea809bcf599141406
W20260810 06:48:04.397346 840 leader_election.cc:341] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15c3cdf8179e4ba29bc934f17f7d0bdc (127.15.232.194:33721): Not found: Tablet not found: 6fd316ff44b843aea809bcf599141406
I20260810 06:48:04.397378 840 leader_election.cc:302] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc35392c1970450587c602d7bf716f80; no voters: 15c3cdf8179e4ba29bc934f17f7d0bdc, 30e4bfb9260c4b4c8c6c6c2482be8865
I20260810 06:48:04.397478 1321 raft_consensus.cc:2726] T 6fd316ff44b843aea809bcf599141406 P cc35392c1970450587c602d7bf716f80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:48:04.576239 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:04.575991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:58931)
0810 06:48:04.576138 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:04.576188 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
I20260810 06:48:04.634143 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:04.633817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:35653)
0810 06:48:04.634041 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:04.634094 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":110,"threads_started":1}
I20260810 06:48:04.772763 1326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:04.772525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:33995)
0810 06:48:04.772667 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:04.772715 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260810 06:48:05.577284 1327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:05.576965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:58611)
0810 06:48:05.577110 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:05.577231 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260810 06:48:05.635107 1328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:05.634836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:44975)
0810 06:48:05.635007 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:05.635056 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
I20260810 06:48:05.773589 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:05.773306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:50175)
0810 06:48:05.773482 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:05.773537 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":100,"threads_started":1}
I20260810 06:48:06.578189 1330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:06.577891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:44177)
0810 06:48:06.578084 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:06.578134 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":95,"threads_started":1}
I20260810 06:48:06.636215 1331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:06.635874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:44427)
0810 06:48:06.636071 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:06.636125 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":83,"threads_started":1}
I20260810 06:48:06.774434 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:06.774181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:51099)
0810 06:48:06.774336 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:06.774382 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260810 06:48:07.579007 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:07.578765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.193:50077)
0810 06:48:07.578917 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:07.578960 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
I20260810 06:48:07.637221 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:07.636927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.194:32965)
0810 06:48:07.637073 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:07.637116 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260810 06:48:07.767613 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 789
I20260810 06:48:07.775306 1335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:07.775051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:42187 (local address 127.15.232.195:36389)
0810 06:48:07.775201 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:07.775260 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260810 06:48:07.776536 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 970
I20260810 06:48:07.782927 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 1127
2026-08-10T06:48:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18693 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:48:07 dist-test-slave-vxj2 krb5kdc[1338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:48:07 dist-test-slave-vxj2 krb5kdc[1338](info): set up 2 sockets
Aug 10 06:48:07 dist-test-slave-vxj2 krb5kdc[1338](info): commencing operation
krb5kdc: starting...
W20260810 06:48:08.943414 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:48:09 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:48:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:48:09.076964 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:41517
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:41517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:09.163339 1353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:09.163616 1353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:09.163658 1353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:09.165223 1353 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:48:09.165325 1353 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:48:09.165350 1353 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:48:09.165370 1353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:09.165402 1353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:09.165432 1353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:09.165462 1353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:09.165485 1353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:09.167282 1353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:41517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:41517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:48:09.167649 1353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:09.167929 1353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:09.173640 1358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:09.173676 1360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:09.173712 1353 server_base.cc:1034] running on GCE node
W20260810 06:48:09.173682 1363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:09.174360 1353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:09.174664 1353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:09.175844 1353 hybrid_clock.cc:648] HybridClock initialized: now 1786344489175817 us; error 32 us; skew 500 ppm
Aug 10 06:48:09 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344489, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:09.178733 1353 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:48:09.179258 1353 webserver.cc:460] Webserver started at http://127.15.232.254:42287/ using document root <none> and password file <none>
I20260810 06:48:09.179517 1353 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:09.179566 1353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:09.179705 1353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:09.180848 1353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "550492d95fcf403ba087a133e7b832c8"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "bdcfb4ffb640c47acfe6cb19d645d416"
server_key_iv: "a29e7fa1ab535db911a4f2ae89069d89"
server_key_version: "encryptionkey@0"
I20260810 06:48:09.181262 1353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "550492d95fcf403ba087a133e7b832c8"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "bdcfb4ffb640c47acfe6cb19d645d416"
server_key_iv: "a29e7fa1ab535db911a4f2ae89069d89"
server_key_version: "encryptionkey@0"
I20260810 06:48:09.182745 1353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:48:09.183621 1371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:09.183890 1353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:09.184036 1353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "550492d95fcf403ba087a133e7b832c8"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "bdcfb4ffb640c47acfe6cb19d645d416"
server_key_iv: "a29e7fa1ab535db911a4f2ae89069d89"
server_key_version: "encryptionkey@0"
I20260810 06:48:09.184130 1353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:09.198901 1353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:09.201262 1353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:09.201473 1353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:09.205714 1353 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:41517
I20260810 06:48:09.205919 1432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:41517 every 8 connection(s)
I20260810 06:48:09.206178 1353 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:09.207213 1433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:09.209729 1433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Bootstrap starting.
I20260810 06:48:09.210389 1433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:09.210711 1433 log.cc:824] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:09.211454 1433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: No bootstrap required, opened a new log
I20260810 06:48:09.213025 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 1353
I20260810 06:48:09.213176 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:48:09.213392 16291 external_mini_cluster.cc:1428] Setting key 97e59ed59c6aee50e5cce133fc6ffe3c
I20260810 06:48:09.213460 1433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } }
I20260810 06:48:09.213677 1433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:09.213729 1433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550492d95fcf403ba087a133e7b832c8, State: Initialized, Role: FOLLOWER
I20260810 06:48:09.213867 1433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } }
I20260810 06:48:09.213963 1433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:09.214038 1433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:09.214160 1433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:09.215094 1433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } }
I20260810 06:48:09.215220 1433 leader_election.cc:302] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 550492d95fcf403ba087a133e7b832c8; no voters:
I20260810 06:48:09.215425 1433 leader_election.cc:288] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:09.215456 1438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:09.215567 1438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 1 LEADER]: Becoming Leader. State: Replica: 550492d95fcf403ba087a133e7b832c8, State: Running, Role: LEADER
I20260810 06:48:09.215708 1433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:09.215710 1438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } }
I20260810 06:48:09.216284 1439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "550492d95fcf403ba087a133e7b832c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } } }
I20260810 06:48:09.216316 1441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 550492d95fcf403ba087a133e7b832c8. Latest consensus state: current_term: 1 leader_uuid: "550492d95fcf403ba087a133e7b832c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } } }
I20260810 06:48:09.216441 1439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:09.216549 1441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:09.216967 1449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:09.217556 1449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:09.219296 1449 catalog_manager.cc:1347] Generated new cluster ID: 6416e738be654f2f847fec6ce4b9f0ff
I20260810 06:48:09.219352 1449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:48:09 dist-test-slave-vxj2 krb5kdc[1338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:48:09.226306 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.213922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59406 (local address 127.15.232.254:41517)
0810 06:48:09.214147 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:09.214159 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:09.214200 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:48:09.214350 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:09.214356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:09.214421 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:09.214558 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:09.215324 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.216754 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.218084 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.218733 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.218866 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.222103 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:09.222108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:09.222108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:09.222124 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:09.224425 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:09.224861 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:09.224865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:09.224865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:09.225017 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:09.225246 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:09.225247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:09.225247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:09.225624 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:09.225807 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:09.225923 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 06:48:09.226086 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":112,"threads_started":1}
I20260810 06:48:09.231304 1449 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:09.232147 1449 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:09.246589 1449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Generated new TSK 0
I20260810 06:48:09.246853 1449 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:48:09.280972 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41517
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:09.366529 1464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:09.366781 1464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:09.366811 1464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:09.368269 1464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:09.368330 1464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:09.368361 1464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:48:09.370038 1464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:48:09.370314 1464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:09.370579 1464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:09.376158 1470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:09.376226 1471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:09.376158 1473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:09.376497 1464 server_base.cc:1034] running on GCE node
I20260810 06:48:09.376701 1464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:09.376976 1464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:09.378162 1464 hybrid_clock.cc:648] HybridClock initialized: now 1786344489378152 us; error 42 us; skew 500 ppm
Aug 10 06:48:09 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344489, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:09.381443 1464 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:48:09.382136 1464 webserver.cc:460] Webserver started at http://127.15.232.193:33475/ using document root <none> and password file <none>
I20260810 06:48:09.382491 1464 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:09.382577 1464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:09.382750 1464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:09.384590 1464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "a989708ef3d54217a22640b3c6041182"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "53623c9ab52f89312a10822169dce6a9"
server_key_iv: "09567922ae2e4118202cf4534e773046"
server_key_version: "encryptionkey@0"
I20260810 06:48:09.385121 1464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "a989708ef3d54217a22640b3c6041182"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "53623c9ab52f89312a10822169dce6a9"
server_key_iv: "09567922ae2e4118202cf4534e773046"
server_key_version: "encryptionkey@0"
I20260810 06:48:09.387233 1464 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260810 06:48:09.388324 1481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:09.388564 1464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:09.388674 1464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "a989708ef3d54217a22640b3c6041182"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "53623c9ab52f89312a10822169dce6a9"
server_key_iv: "09567922ae2e4118202cf4534e773046"
server_key_version: "encryptionkey@0"
I20260810 06:48:09.388757 1464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:09.396557 1464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:09.398880 1464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:09.399061 1464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:09.399302 1464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:09.399641 1464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:09.399703 1464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:09.399745 1464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:09.399776 1464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:09.405936 1464 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:36535
I20260810 06:48:09.406054 1613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:36535 every 8 connection(s)
I20260810 06:48:09.406378 1464 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:48:09.407205 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 1464
I20260810 06:48:09.407341 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:48:09.407586 16291 external_mini_cluster.cc:1428] Setting key 794816b09f05a31b003aa80b43f6cc83
Aug 10 06:48:09 dist-test-slave-vxj2 krb5kdc[1338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344489, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
I20260810 06:48:09.417476 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.407330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:56483 (local address 127.15.232.254:41517)
0810 06:48:09.407865 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:09.407867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:09.407880 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:48:09.408447 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:09.408449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:09.408464 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:09.408519 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:09.409623 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.410146 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.411195 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.411446 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.411532 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.414237 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:09.414239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:09.414239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:09.414248 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:09.415805 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:09.416274 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:09.416276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:09.416276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:09.416338 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:09.416704 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:09.416705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:09.416706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:09.416835 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:09.416909 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:09.417273 (+ 364us) server_negotiation.cc:299] Negotiation successful
0810 06:48:09.417365 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442}
I20260810 06:48:09.417692 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.407336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:56483)
0810 06:48:09.407594 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:09.407652 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:09.408339 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:09.408604 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:09.408608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:09.409303 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:09.409518 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.409524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.410255 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.410256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.411094 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.411096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.411542 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.411544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.411683 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.412287 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:09.412296 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:09.414123 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:09.415894 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:09.415903 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:09.415906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:09.416192 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:09.416413 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:09.416414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:09.416414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:09.416640 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:09.416911 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:09.416926 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:09.417079 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 06:48:09.417250 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 06:48:09.417432 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":136,"threads_started":1}
I20260810 06:48:09.418258 1614 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41517
I20260810 06:48:09.418397 1614 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:09.418620 1614 heartbeater.cc:511] Master 127.15.232.254:41517 requested a full tablet report, sending...
I20260810 06:48:09.419287 1392 ts_manager.cc:194] Registered new tserver with Master: a989708ef3d54217a22640b3c6041182 (127.15.232.193:36535)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
I20260810 06:48:09.419911 1392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:56483
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:48:09.437250 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41517
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:09.527416 1619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:09.527642 1619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:09.527667 1619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:09.528992 1619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:09.529045 1619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:09.529074 1619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:48:09.530627 1619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:48:09.530897 1619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:09.531149 1619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:09.536546 1625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:09.536547 1628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:09.536792 1619 server_base.cc:1034] running on GCE node
W20260810 06:48:09.536805 1624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:09.537073 1619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:09.537369 1619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:09.538540 1619 hybrid_clock.cc:648] HybridClock initialized: now 1786344489538530 us; error 31 us; skew 500 ppm
Aug 10 06:48:09 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344489, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:09.541203 1619 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:48:09.541621 1619 webserver.cc:460] Webserver started at http://127.15.232.194:34579/ using document root <none> and password file <none>
I20260810 06:48:09.541833 1619 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:09.541877 1619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:09.541985 1619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:09.543033 1619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "1c188491c2a140c38f1134be4a9050c6"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "585c90f55d57a66ed189b1434e7a9bd4"
server_key_iv: "86ba1c625c5037b5897101981a7989ac"
server_key_version: "encryptionkey@0"
I20260810 06:48:09.543342 1619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "1c188491c2a140c38f1134be4a9050c6"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "585c90f55d57a66ed189b1434e7a9bd4"
server_key_iv: "86ba1c625c5037b5897101981a7989ac"
server_key_version: "encryptionkey@0"
I20260810 06:48:09.544839 1619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:48:09.545729 1636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:09.545980 1619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:48:09.546093 1619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "1c188491c2a140c38f1134be4a9050c6"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "585c90f55d57a66ed189b1434e7a9bd4"
server_key_iv: "86ba1c625c5037b5897101981a7989ac"
server_key_version: "encryptionkey@0"
I20260810 06:48:09.546161 1619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:09.565485 1619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:09.568240 1619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:09.568449 1619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:09.568763 1619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:09.569077 1619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:09.569113 1619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:09.569139 1619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:09.569231 1619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:09.575872 1619 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:36317
I20260810 06:48:09.575934 1761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:36317 every 8 connection(s)
I20260810 06:48:09.576296 1619 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:48:09 dist-test-slave-vxj2 krb5kdc[1338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344489, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:48:09.583577 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 1619
I20260810 06:48:09.583698 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:48:09.583891 16291 external_mini_cluster.cc:1428] Setting key 7276badf777d8c44fba39b696450b1fe
I20260810 06:48:09.587004 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.577136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:52613 (local address 127.15.232.254:41517)
0810 06:48:09.577222 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:09.577225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:09.577556 (+ 331us) server_negotiation.cc:407] Connection header received
0810 06:48:09.578111 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:09.578113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:09.578125 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:09.578188 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:09.579197 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.579704 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.580838 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.581040 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.581099 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.583330 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:09.583333 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:09.583333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:09.583340 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:09.585223 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:09.585767 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:09.585768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:09.585769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:09.585825 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:09.586127 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:09.586128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:09.586128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:09.586240 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:09.586296 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:09.586710 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 06:48:09.586821 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:48:09.587088 1764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.577127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:52613)
0810 06:48:09.577364 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:09.577433 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:09.577978 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:09.578243 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:09.578246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:09.578864 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:09.579085 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.579090 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.579821 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.579823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.580746 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.580749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.581124 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.581125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.581284 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.581785 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:09.581794 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:09.583235 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:09.585311 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:09.585321 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:09.585324 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:09.585683 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:09.585896 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:09.585897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:09.585897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:09.586060 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:09.586310 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:09.586317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:09.586535 (+ 218us) client_negotiation.cc:769] Sending connection context
0810 06:48:09.586703 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 06:48:09.586873 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":148,"threads_started":1}
I20260810 06:48:09.587689 1762 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41517
I20260810 06:48:09.587831 1762 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:09.588040 1762 heartbeater.cc:511] Master 127.15.232.254:41517 requested a full tablet report, sending...
I20260810 06:48:09.588551 1392 ts_manager.cc:194] Registered new tserver with Master: 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317)
I20260810 06:48:09.589135 1392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:52613
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:48:09.612702 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41517
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:09.698868 1767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:09.699144 1767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:09.699190 1767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:09.700455 1767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:09.700529 1767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:09.700577 1767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:48:09.702253 1767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:48:09.702569 1767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:09.702822 1767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:09.708289 1772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:09.708305 1775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:09.708308 1773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:09.708546 1767 server_base.cc:1034] running on GCE node
I20260810 06:48:09.708789 1767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:09.709053 1767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:09.710202 1767 hybrid_clock.cc:648] HybridClock initialized: now 1786344489710190 us; error 28 us; skew 500 ppm
Aug 10 06:48:09 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344489, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:09.712984 1767 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:48:09.713537 1767 webserver.cc:460] Webserver started at http://127.15.232.195:35083/ using document root <none> and password file <none>
I20260810 06:48:09.713793 1767 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:09.713842 1767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:09.714020 1767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:09.715130 1767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "af5e1eb6498741469fdb14401636c63b"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "a7a2122bae589af00b85cbb8d46bc89f"
server_key_iv: "0107f27512b96ccd98e91c54704c5c14"
server_key_version: "encryptionkey@0"
I20260810 06:48:09.715559 1767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "af5e1eb6498741469fdb14401636c63b"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "a7a2122bae589af00b85cbb8d46bc89f"
server_key_iv: "0107f27512b96ccd98e91c54704c5c14"
server_key_version: "encryptionkey@0"
I20260810 06:48:09.717219 1767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:48:09.718119 1783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:09.718274 1767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:09.718362 1767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "af5e1eb6498741469fdb14401636c63b"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "a7a2122bae589af00b85cbb8d46bc89f"
server_key_iv: "0107f27512b96ccd98e91c54704c5c14"
server_key_version: "encryptionkey@0"
I20260810 06:48:09.718420 1767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:09.734807 1767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:09.737311 1767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:09.737440 1767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:09.737645 1767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:09.737982 1767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:09.738022 1767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:09.738046 1767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:09.738058 1767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:09.744633 1767 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:40573
I20260810 06:48:09.744683 1915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:40573 every 8 connection(s)
I20260810 06:48:09.745056 1767 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:48:09.748929 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 1767
I20260810 06:48:09.749035 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:48:09.749235 16291 external_mini_cluster.cc:1428] Setting key 8d8838018472b0da21afe192fe41e2b5
Aug 10 06:48:09 dist-test-slave-vxj2 krb5kdc[1338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344489, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:48:09.755987 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.745966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:37389 (local address 127.15.232.254:41517)
0810 06:48:09.746051 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:09.746053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:09.746210 (+ 157us) server_negotiation.cc:407] Connection header received
0810 06:48:09.746789 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:09.746791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:09.746802 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:09.746871 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:09.748003 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.748509 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.750152 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.750453 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.750526 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.752747 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:09.752749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:09.752749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:09.752757 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:09.754415 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:09.754906 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:09.754907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:09.754907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:09.754971 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:09.755300 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:09.755301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:09.755301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:09.755433 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:09.755499 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:09.755836 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 06:48:09.755919 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:48:09.756055 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.745880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:37389)
0810 06:48:09.746060 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:09.746107 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:09.746658 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:09.747001 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:09.747006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:09.747667 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:09.747879 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.747884 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.748605 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.748606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.750059 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.750063 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.750538 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.750539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.750656 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.751193 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:09.751201 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:09.752646 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:09.754518 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:09.754526 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:09.754528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:09.754812 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:09.755061 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:09.755062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:09.755062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:09.755204 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:09.755504 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:09.755510 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:09.755640 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 06:48:09.755731 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 06:48:09.755875 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
I20260810 06:48:09.756659 1917 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41517
I20260810 06:48:09.756763 1917 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:09.756968 1917 heartbeater.cc:511] Master 127.15.232.254:41517 requested a full tablet report, sending...
I20260810 06:48:09.757462 1392 ts_manager.cc:194] Registered new tserver with Master: af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573)
I20260810 06:48:09.757813 1392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:37389
I20260810 06:48:09.761238 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:48:09.768785 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.762665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59412 (local address 127.15.232.254:41517)
0810 06:48:09.762731 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:09.762733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:09.762857 (+ 124us) server_negotiation.cc:407] Connection header received
0810 06:48:09.762952 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:09.762953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:09.762961 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:09.763008 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:09.763561 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.764065 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.764762 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.764969 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.765024 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.766282 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:09.766283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:09.766283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:09.766289 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:09.767625 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:09.768032 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:09.768033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:09.768033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:09.768086 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:09.768306 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:09.768307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:09.768307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:09.768455 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:09.768532 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:09.768630 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 06:48:09.768691 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:48:09.770597 1392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:48:09.772249 1392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:48:09.779139 1932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.774738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:40573 (local address 127.0.0.1:43698)
0810 06:48:09.774919 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:09.774963 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:09.775037 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:09.775565 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:09.775568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:09.775570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:09.775661 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.775664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.777034 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.777035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.778112 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.778114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.778802 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.778803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.778883 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.778898 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 06:48:09.778976 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:48:09.779031 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":127,"threads_started":1}
I20260810 06:48:09.779315 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.775148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54410 (local address 127.15.232.193:36535)
0810 06:48:09.775349 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:09.775352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:09.775363 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:48:09.775399 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:09.775400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:09.775414 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:09.775510 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:09.775746 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.776600 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.778171 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.778904 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.778987 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.779047 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:09.779126 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 06:48:09.779177 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":88,"threads_started":1}
I20260810 06:48:09.779346 1935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.774934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43698 (local address 127.15.232.195:40573)
0810 06:48:09.775365 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:09.775368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:09.775398 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:48:09.775442 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:09.775444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:09.775462 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:09.775543 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:09.775754 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.776926 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.778206 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.778707 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.778804 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.778871 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:09.779188 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 06:48:09.779240 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":96,"threads_started":1}
I20260810 06:48:09.779561 1936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.775012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43466 (local address 127.15.232.194:36317)
0810 06:48:09.775211 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:09.775216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:09.775775 (+ 559us) server_negotiation.cc:407] Connection header received
0810 06:48:09.775826 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:09.775828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:09.775843 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:09.775937 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:09.776072 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.777122 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.778279 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.778974 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.779037 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.779083 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:09.779452 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 06:48:09.779503 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":82,"threads_started":1}
I20260810 06:48:09.779381 1934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.774746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:36317 (local address 127.0.0.1:43466)
0810 06:48:09.775655 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:09.775666 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:09.775714 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:09.775945 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:09.775946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:09.775949 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:09.775996 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.775997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.777256 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.777257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.778179 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.778183 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.779120 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.779122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.779181 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.779186 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:48:09.779256 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:48:09.779297 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"thread_start_us":96,"threads_started":1}
I20260810 06:48:09.779959 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.774738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:36535 (local address 127.0.0.1:54410)
0810 06:48:09.774990 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:09.774999 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:09.775041 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:09.775602 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:09.775603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:09.775605 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:09.775662 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.775664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.776713 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.776715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.778073 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.778076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.778982 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.778983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.779051 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.779054 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:48:09.779108 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 06:48:09.779140 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":45,"thread_start_us":47,"threads_started":1}
I20260810 06:48:09.780511 1689 tablet_service.cc:1467] Processing CreateTablet for tablet 2786d5d4e7be4835bc69176721b84ca4 (DEFAULT_TABLE table=test-table [id=11c4b13dc19f4e9081262b2c3d44eaf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:09.780686 1537 tablet_service.cc:1467] Processing CreateTablet for tablet 2786d5d4e7be4835bc69176721b84ca4 (DEFAULT_TABLE table=test-table [id=11c4b13dc19f4e9081262b2c3d44eaf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:09.780929 1689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2786d5d4e7be4835bc69176721b84ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:09.780989 1537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2786d5d4e7be4835bc69176721b84ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:09.781560 1841 tablet_service.cc:1467] Processing CreateTablet for tablet 2786d5d4e7be4835bc69176721b84ca4 (DEFAULT_TABLE table=test-table [id=11c4b13dc19f4e9081262b2c3d44eaf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:09.781903 1841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2786d5d4e7be4835bc69176721b84ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:09.784484 1938 tablet_bootstrap.cc:492] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b: Bootstrap starting.
I20260810 06:48:09.785058 1939 tablet_bootstrap.cc:492] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Bootstrap starting.
I20260810 06:48:09.785284 1938 tablet_bootstrap.cc:654] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:09.785817 1939 tablet_bootstrap.cc:654] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:09.785941 1938 log.cc:824] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:09.786396 1939 log.cc:824] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:09.786792 1940 tablet_bootstrap.cc:492] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6: Bootstrap starting.
I20260810 06:48:09.786881 1938 tablet_bootstrap.cc:492] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b: No bootstrap required, opened a new log
I20260810 06:48:09.786943 1938 ts_tablet_manager.cc:1397] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 06:48:09.787267 1939 tablet_bootstrap.cc:492] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: No bootstrap required, opened a new log
I20260810 06:48:09.787352 1939 ts_tablet_manager.cc:1397] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:48:09.787456 1940 tablet_bootstrap.cc:654] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:09.788038 1940 log.cc:824] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:09.788836 1940 tablet_bootstrap.cc:492] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6: No bootstrap required, opened a new log
I20260810 06:48:09.788928 1940 ts_tablet_manager.cc:1397] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:48:09.789126 1938 raft_consensus.cc:348] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } } peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } }
I20260810 06:48:09.789377 1938 raft_consensus.cc:374] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:09.789427 1938 raft_consensus.cc:729] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af5e1eb6498741469fdb14401636c63b, State: Initialized, Role: FOLLOWER
I20260810 06:48:09.789552 1938 consensus_queue.cc:260] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } } peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } }
I20260810 06:48:09.789602 1939 raft_consensus.cc:348] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } } peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } }
I20260810 06:48:09.789800 1939 raft_consensus.cc:374] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:09.789849 1939 raft_consensus.cc:729] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a989708ef3d54217a22640b3c6041182, State: Initialized, Role: FOLLOWER
I20260810 06:48:09.789878 1938 ts_tablet_manager.cc:1428] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 06:48:09.790043 1917 heartbeater.cc:503] Master 127.15.232.254:41517 was elected leader, sending a full tablet report...
I20260810 06:48:09.790001 1939 consensus_queue.cc:260] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } } peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } }
I20260810 06:48:09.790282 1939 ts_tablet_manager.cc:1428] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260810 06:48:09.790449 1614 heartbeater.cc:503] Master 127.15.232.254:41517 was elected leader, sending a full tablet report...
I20260810 06:48:09.791231 1940 raft_consensus.cc:348] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } } peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } }
I20260810 06:48:09.791424 1940 raft_consensus.cc:374] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:09.791473 1940 raft_consensus.cc:729] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c188491c2a140c38f1134be4a9050c6, State: Initialized, Role: FOLLOWER
I20260810 06:48:09.791594 1940 consensus_queue.cc:260] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } } peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } }
I20260810 06:48:09.791921 1940 ts_tablet_manager.cc:1428] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:48:09.791934 1762 heartbeater.cc:503] Master 127.15.232.254:41517 was elected leader, sending a full tablet report...
I20260810 06:48:09.792299 1945 raft_consensus.cc:482] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:09.792438 1945 raft_consensus.cc:504] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } } peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } }
I20260810 06:48:09.792906 1945 leader_election.cc:288] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573), 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317)
I20260810 06:48:09.795931 1936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.793081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:45335 (local address 127.15.232.194:36317)
0810 06:48:09.793337 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:09.793340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:09.793347 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:48:09.793370 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:09.793371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:09.793382 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:09.793429 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:09.793627 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.794165 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.795171 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.795571 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.795629 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.795664 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:09.795815 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 06:48:09.795862 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215}
I20260810 06:48:09.795931 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.793054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:36317 (local address 127.15.232.193:45335)
0810 06:48:09.793129 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:09.793143 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:09.793214 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:09.793453 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:09.793454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:09.793457 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:09.793535 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.793537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.794298 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.794299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.795074 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.795076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.795653 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.795654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.795735 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.795739 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:48:09.795792 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 06:48:09.795824 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":36}
I20260810 06:48:09.795990 1935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.793264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:42907 (local address 127.15.232.195:40573)
0810 06:48:09.793315 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:09.793317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:09.793326 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:48:09.793421 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:09.793423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:09.793433 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:09.793482 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:09.793647 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.794144 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.795198 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.795580 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.795670 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.795679 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:09.795874 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 06:48:09.795916 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:48:09.796021 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.793055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:40573 (local address 127.15.232.193:42907)
0810 06:48:09.793236 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:09.793246 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:09.793350 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:09.793501 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:09.793501 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:09.793503 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:09.793574 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.793575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.794235 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.794236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.795083 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:09.795085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.795689 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.795690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:09.795790 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.795795 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:48:09.795855 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 06:48:09.795892 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":50,"thread_start_us":108,"threads_started":1}
I20260810 06:48:09.796268 1710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2786d5d4e7be4835bc69176721b84ca4" candidate_uuid: "a989708ef3d54217a22640b3c6041182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c188491c2a140c38f1134be4a9050c6" is_pre_election: true
I20260810 06:48:09.796336 1863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2786d5d4e7be4835bc69176721b84ca4" candidate_uuid: "a989708ef3d54217a22640b3c6041182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af5e1eb6498741469fdb14401636c63b" is_pre_election: true
I20260810 06:48:09.796420 1710 raft_consensus.cc:2454] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a989708ef3d54217a22640b3c6041182 in term 0.
I20260810 06:48:09.796473 1863 raft_consensus.cc:2454] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a989708ef3d54217a22640b3c6041182 in term 0.
I20260810 06:48:09.796676 1482 leader_election.cc:302] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a989708ef3d54217a22640b3c6041182, af5e1eb6498741469fdb14401636c63b; no voters:
I20260810 06:48:09.796828 1945 raft_consensus.cc:2781] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:48:09.796883 1945 raft_consensus.cc:482] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:48:09.796933 1945 raft_consensus.cc:3037] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:09.797729 1945 raft_consensus.cc:504] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } } peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } }
I20260810 06:48:09.797899 1945 leader_election.cc:288] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 1 election: Requested vote from peers af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573), 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317)
I20260810 06:48:09.798079 1863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2786d5d4e7be4835bc69176721b84ca4" candidate_uuid: "a989708ef3d54217a22640b3c6041182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af5e1eb6498741469fdb14401636c63b"
I20260810 06:48:09.798079 1710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2786d5d4e7be4835bc69176721b84ca4" candidate_uuid: "a989708ef3d54217a22640b3c6041182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c188491c2a140c38f1134be4a9050c6"
I20260810 06:48:09.798159 1710 raft_consensus.cc:3037] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:09.798161 1863 raft_consensus.cc:3037] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:09.799072 1710 raft_consensus.cc:2454] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a989708ef3d54217a22640b3c6041182 in term 1.
I20260810 06:48:09.799075 1863 raft_consensus.cc:2454] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a989708ef3d54217a22640b3c6041182 in term 1.
I20260810 06:48:09.799232 1482 leader_election.cc:302] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a989708ef3d54217a22640b3c6041182, af5e1eb6498741469fdb14401636c63b; no voters:
I20260810 06:48:09.799309 1945 raft_consensus.cc:2781] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:09.799467 1945 raft_consensus.cc:686] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 1 LEADER]: Becoming Leader. State: Replica: a989708ef3d54217a22640b3c6041182, State: Running, Role: LEADER
I20260810 06:48:09.799595 1945 consensus_queue.cc:237] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } } peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } }
I20260810 06:48:09.800498 1391 catalog_manager.cc:5495] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 reported cstate change: term changed from 0 to 1, leader changed from <none> to a989708ef3d54217a22640b3c6041182 (127.15.232.193). New cstate: current_term: 1 leader_uuid: "a989708ef3d54217a22640b3c6041182" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } health_report { overall_health: UNKNOWN } } }
I20260810 06:48:09.809239 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.806636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54420 (local address 127.15.232.193:36535)
0810 06:48:09.806705 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:09.806708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:09.806717 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:48:09.806777 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:09.806778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:09.806805 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:09.806886 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:48:09.807041 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.807537 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.808372 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:09.808578 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:09.808655 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:09.808831 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:48:09.808946 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:48:09.809040 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:09.809071 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 06:48:09.809091 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:48:09.811138 1710 raft_consensus.cc:1261] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6 [term 1 FOLLOWER]: Refusing update from remote peer a989708ef3d54217a22640b3c6041182: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:09.811174 1863 raft_consensus.cc:1261] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b [term 1 FOLLOWER]: Refusing update from remote peer a989708ef3d54217a22640b3c6041182: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:09.811503 1945 consensus_queue.cc:1035] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:09.811651 1948 consensus_queue.cc:1035] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:09.814456 1952 mvcc.cc:205] Tried to move back new op lower bound from 7316867030263631872 to 7316867030219870208. Current Snapshot: MvccSnapshot[applied={T|T < 7316867030263631872}]
I20260810 06:48:09.816461 1391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:09.816574 1391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:09.817356 1391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Sending DeleteTablet for 3 replicas of tablet 2786d5d4e7be4835bc69176721b84ca4
I20260810 06:48:09.817705 1689 tablet_service.cc:1514] Processing DeleteTablet for tablet 2786d5d4e7be4835bc69176721b84ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:09 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:43466
I20260810 06:48:09.817750 1841 tablet_service.cc:1514] Processing DeleteTablet for tablet 2786d5d4e7be4835bc69176721b84ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:09 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:43698
I20260810 06:48:09.817976 1537 tablet_service.cc:1514] Processing DeleteTablet for tablet 2786d5d4e7be4835bc69176721b84ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:09 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:54410
I20260810 06:48:09.818041 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 1464
I20260810 06:48:09.818018 1962 tablet_replica.cc:330] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b: stopping tablet replica
I20260810 06:48:09.818223 1962 raft_consensus.cc:2229] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:09.818399 1962 raft_consensus.cc:2258] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:09.819482 1961 tablet_replica.cc:330] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6: stopping tablet replica
I20260810 06:48:09.819594 1961 raft_consensus.cc:2229] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:09.819696 1961 raft_consensus.cc:2258] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:09.819943 1962 ts_tablet_manager.cc:1905] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:09.820717 1961 ts_tablet_manager.cc:1905] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:09.822613 1962 ts_tablet_manager.cc:1918] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:09.822641 1961 ts_tablet_manager.cc:1918] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:09.822727 1961 log.cc:1196] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/2786d5d4e7be4835bc69176721b84ca4
I20260810 06:48:09.822726 1962 log.cc:1196] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/2786d5d4e7be4835bc69176721b84ca4
I20260810 06:48:09.823071 1961 ts_tablet_manager.cc:1939] T 2786d5d4e7be4835bc69176721b84ca4 P 1c188491c2a140c38f1134be4a9050c6: Deleting consensus metadata
I20260810 06:48:09.823118 1962 ts_tablet_manager.cc:1939] T 2786d5d4e7be4835bc69176721b84ca4 P af5e1eb6498741469fdb14401636c63b: Deleting consensus metadata
I20260810 06:48:09.823580 1373 catalog_manager.cc:4841] TS 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317): tablet 2786d5d4e7be4835bc69176721b84ca4 (table test-table [id=11c4b13dc19f4e9081262b2c3d44eaf0]) successfully deleted
I20260810 06:48:09.825968 1372 catalog_manager.cc:4841] TS af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573): tablet 2786d5d4e7be4835bc69176721b84ca4 (table test-table [id=11c4b13dc19f4e9081262b2c3d44eaf0]) successfully deleted
W20260810 06:48:09.830312 1375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:36535 (error 108)
I20260810 06:48:09.830560 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 1619
I20260810 06:48:09.832319 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.832144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:36535 (local address 127.0.0.1:54436)
0810 06:48:09.832219 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:09.832272 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:36535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 06:48:09.832568 1375 catalog_manager.cc:4584] TS a989708ef3d54217a22640b3c6041182 (127.15.232.193:36535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2786d5d4e7be4835bc69176721b84ca4: Network error: Client connection negotiation failed: client connection to 127.15.232.193:36535: connect: Connection refused (error 111)
I20260810 06:48:09.837226 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 1767
I20260810 06:48:09.845358 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 1353
I20260810 06:48:29.853084 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:41517
--webserver_interface=127.15.232.254
--webserver_port=42287
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:41517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:29.939639 1964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:29.939867 1964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:29.939890 1964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:29.941094 1964 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:48:29.941136 1964 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:48:29.941170 1964 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:48:29.941182 1964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:29.941195 1964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:29.941239 1964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:29.941270 1964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:29.941287 1964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:29.942806 1964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:41517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:41517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=42287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:48:29.943073 1964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:29.943321 1964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:29.953563 1970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:29.953584 1971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:29.953616 1964 server_base.cc:1034] running on GCE node
W20260810 06:48:29.953572 1973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:29.954046 1964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:29.954452 1964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:29.955636 1964 hybrid_clock.cc:648] HybridClock initialized: now 1786344509955625 us; error 49 us; skew 500 ppm
Aug 10 06:48:29 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344509, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:29.958590 1964 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:48:29.959148 1964 webserver.cc:460] Webserver started at http://127.15.232.254:42287/ using document root <none> and password file <none>
I20260810 06:48:29.959445 1964 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:29.959542 1964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:29.961109 1964 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:48:29.962000 1981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:29.962193 1964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:29.962297 1964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "550492d95fcf403ba087a133e7b832c8"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "bdcfb4ffb640c47acfe6cb19d645d416"
server_key_iv: "a29e7fa1ab535db911a4f2ae89069d89"
server_key_version: "encryptionkey@0"
I20260810 06:48:29.962733 1964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:29.978003 1964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:29.980437 1964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:29.980566 1964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:29.984742 1964 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:41517
I20260810 06:48:29.984810 2042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:41517 every 8 connection(s)
I20260810 06:48:29.985308 1964 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:29.987056 2043 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:48:29.987831 2043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Bootstrap starting.
I20260810 06:48:29.989313 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 1964
I20260810 06:48:29.989697 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:36535
--local_ip_for_outbound_sockets=127.15.232.193
--tserver_master_addrs=127.15.232.254:41517
--webserver_port=33475
--webserver_interface=127.15.232.193
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:48:29.997864 2043 log.cc:824] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:30.000108 2043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:48:30.000502 2043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Bootstrap complete.
I20260810 06:48:30.003159 2043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } }
I20260810 06:48:30.003448 2043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550492d95fcf403ba087a133e7b832c8, State: Initialized, Role: FOLLOWER
I20260810 06:48:30.003659 2043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } }
I20260810 06:48:30.003732 2043 raft_consensus.cc:388] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:30.003810 2043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:30.003870 2043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:48:30.005105 2043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } }
I20260810 06:48:30.005324 2043 leader_election.cc:302] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 550492d95fcf403ba087a133e7b832c8; no voters:
I20260810 06:48:30.005581 2043 leader_election.cc:288] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:48:30.005626 2049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:48:30.005774 2049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 2 LEADER]: Becoming Leader. State: Replica: 550492d95fcf403ba087a133e7b832c8, State: Running, Role: LEADER
I20260810 06:48:30.005904 2043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:30.005878 2049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } }
I20260810 06:48:30.006217 2051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 550492d95fcf403ba087a133e7b832c8. Latest consensus state: current_term: 2 leader_uuid: "550492d95fcf403ba087a133e7b832c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } } }
I20260810 06:48:30.006449 2051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:30.006392 2050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "550492d95fcf403ba087a133e7b832c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } } }
I20260810 06:48:30.006752 2050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:30.006789 2054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:30.007819 2054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:30.007963 2054 catalog_manager.cc:1259] Loaded cluster ID: 6416e738be654f2f847fec6ce4b9f0ff
I20260810 06:48:30.008270 2054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:30.009272 2054 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:30.009497 2054 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Loaded TSK: 0
I20260810 06:48:30.017534 2054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Generated new TSK 1
I20260810 06:48:30.017681 2054 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:48:30.104789 2046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:30.105032 2046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:30.105059 2046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:30.106428 2046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:30.106490 2046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:30.106523 2046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:48:30.108213 2046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:36535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=33475
--tserver_master_addrs=127.15.232.254:41517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:48:30.108472 2046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:30.108736 2046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:30.114089 2079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:30.114089 2077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:30.114326 2046 server_base.cc:1034] running on GCE node
W20260810 06:48:30.114081 2076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:30.114651 2046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:30.115036 2046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:30.116220 2046 hybrid_clock.cc:648] HybridClock initialized: now 1786344510116198 us; error 40 us; skew 500 ppm
Aug 10 06:48:30 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344510, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:30.119371 2046 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:48:30.119947 2046 webserver.cc:460] Webserver started at http://127.15.232.193:33475/ using document root <none> and password file <none>
I20260810 06:48:30.120257 2046 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:30.120374 2046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:30.122210 2046 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:48:30.123026 2087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:30.123221 2046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:48:30.123356 2046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "a989708ef3d54217a22640b3c6041182"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "53623c9ab52f89312a10822169dce6a9"
server_key_iv: "09567922ae2e4118202cf4534e773046"
server_key_version: "encryptionkey@0"
I20260810 06:48:30.123725 2046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:30.166199 2046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:30.168772 2046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:30.168952 2046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:30.169262 2046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:30.169790 2097 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 06:48:30.171949 2046 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 06:48:30.172048 2046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 06:48:30.172102 2046 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 06:48:30.172804 2046 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 06:48:30.172865 2046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 06:48:30.172922 2097 tablet_bootstrap.cc:492] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Bootstrap starting.
I20260810 06:48:30.180425 2046 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:36535
I20260810 06:48:30.180599 2220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:36535 every 8 connection(s)
I20260810 06:48:30.180900 2046 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:48:30.184327 2097 log.cc:824] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:30.187001 2097 tablet_bootstrap.cc:492] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:48:30.187230 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 2046
I20260810 06:48:30.187402 2097 tablet_bootstrap.cc:492] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Bootstrap complete.
I20260810 06:48:30.187564 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:36317
--local_ip_for_outbound_sockets=127.15.232.194
--tserver_master_addrs=127.15.232.254:41517
--webserver_port=34579
--webserver_interface=127.15.232.194
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:48:30.188557 2097 ts_tablet_manager.cc:1397] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.004s
Aug 10 06:48:30 dist-test-slave-vxj2 krb5kdc[1338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344510, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:48:30.189985 2097 raft_consensus.cc:348] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } } peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } }
I20260810 06:48:30.190371 2097 raft_consensus.cc:729] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a989708ef3d54217a22640b3c6041182, State: Initialized, Role: FOLLOWER
I20260810 06:48:30.190531 2097 consensus_queue.cc:260] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } } peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } }
I20260810 06:48:30.190855 2097 ts_tablet_manager.cc:1428] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
I20260810 06:48:30.194344 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.181703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:43467)
0810 06:48:30.181904 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:30.181956 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:30.182465 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:30.183243 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:30.183247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:30.183546 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:30.183758 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.183764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.185468 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.185469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.186285 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.186287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.187247 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.187248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.187425 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.188161 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:30.188172 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:30.190110 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:30.192615 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:30.192622 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:30.192624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:30.192876 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:30.193224 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:30.193225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:30.193226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:30.193341 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:30.193840 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:30.193846 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:30.193967 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 06:48:30.194060 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 06:48:30.194191 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":120,"threads_started":1}
I20260810 06:48:30.194559 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.181814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:43467 (local address 127.15.232.254:41517)
0810 06:48:30.182649 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:30.182664 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:30.182721 (+ 57us) server_negotiation.cc:407] Connection header received
0810 06:48:30.182913 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:30.182922 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:30.182989 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:30.183181 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:30.183983 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.185322 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.186394 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.187009 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.187131 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.190239 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:30.190243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:30.190244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:30.190257 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:30.192519 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:30.192973 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:30.192978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:30.192979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:30.193115 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:30.193412 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:30.193413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:30.193413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:30.193758 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:30.193859 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:30.194155 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 06:48:30.194323 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":95,"threads_started":1}
I20260810 06:48:30.195276 2221 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41517
I20260810 06:48:30.195394 2221 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:30.195603 2221 heartbeater.cc:511] Master 127.15.232.254:41517 requested a full tablet report, sending...
I20260810 06:48:30.196410 1998 ts_manager.cc:194] Registered new tserver with Master: a989708ef3d54217a22640b3c6041182 (127.15.232.193:36535)
I20260810 06:48:30.197655 1998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:43467
I20260810 06:48:30.198553 2221 heartbeater.cc:503] Master 127.15.232.254:41517 was elected leader, sending a full tablet report...
Aug 10 06:48:30 dist-test-slave-vxj2 krb5kdc[1338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344509, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.193@KRBTEST.COM
I20260810 06:48:30.205870 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.197012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:36535 (local address 127.0.0.1:41436)
0810 06:48:30.197258 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:30.197291 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:30.197352 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:30.197574 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:30.197578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:30.198000 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:30.198066 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.198069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.199436 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.199436 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.200041 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.200043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.200695 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.200696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.200754 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.201358 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:30.201371 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:30.202691 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:30.204646 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:30.204653 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:30.204655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:30.204872 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:30.205109 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:30.205110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:30.205110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:30.205237 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:30.205559 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:30.205566 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:30.205613 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 06:48:30.205674 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 06:48:30.205782 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":94,"threads_started":1}
I20260810 06:48:30.205955 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.197101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.15.232.193:36535)
0810 06:48:30.197329 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:30.197333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:30.197345 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:48:30.197452 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:30.197455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:30.197474 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:30.197559 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:30.198156 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.199351 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.200129 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.200614 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.200689 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.202770 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:30.202779 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:30.202779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:30.202802 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:30.204554 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:30.204957 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:30.204960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:30.204961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:30.205037 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:30.205302 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:30.205303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:30.205303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:30.205490 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:30.205576 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:30.205747 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 06:48:30.205848 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":195,"thread_start_us":134,"threads_started":1}
I20260810 06:48:30.206327 2145 tablet_service.cc:1514] Processing DeleteTablet for tablet 2786d5d4e7be4835bc69176721b84ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:09 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:41436
I20260810 06:48:30.207005 2230 tablet_replica.cc:330] stopping tablet replica
I20260810 06:48:30.207139 2230 raft_consensus.cc:2229] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:30.207355 2230 raft_consensus.cc:2258] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:30.207734 2230 ts_tablet_manager.cc:1905] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:30.210040 2230 ts_tablet_manager.cc:1918] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:30.210139 2230 log.cc:1196] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal/wals/2786d5d4e7be4835bc69176721b84ca4
I20260810 06:48:30.210402 2230 ts_tablet_manager.cc:1939] T 2786d5d4e7be4835bc69176721b84ca4 P a989708ef3d54217a22640b3c6041182: Deleting consensus metadata
I20260810 06:48:30.210951 1985 catalog_manager.cc:4841] TS a989708ef3d54217a22640b3c6041182 (127.15.232.193:36535): tablet 2786d5d4e7be4835bc69176721b84ca4 (table test-table [id=11c4b13dc19f4e9081262b2c3d44eaf0]) successfully deleted
W20260810 06:48:30.278496 2226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:30.278726 2226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:30.278752 2226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:30.279999 2226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:30.280048 2226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:30.280076 2226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:48:30.281646 2226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:36317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=34579
--tserver_master_addrs=127.15.232.254:41517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:48:30.281898 2226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:30.282158 2226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:30.287544 2238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:30.287544 2235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:30.287616 2236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:30.287770 2226 server_base.cc:1034] running on GCE node
I20260810 06:48:30.288125 2226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:30.288424 2226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:30.289599 2226 hybrid_clock.cc:648] HybridClock initialized: now 1786344510289587 us; error 46 us; skew 500 ppm
Aug 10 06:48:30 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344510, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:30.292387 2226 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:48:30.292925 2226 webserver.cc:460] Webserver started at http://127.15.232.194:34579/ using document root <none> and password file <none>
I20260810 06:48:30.293221 2226 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:30.293309 2226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:30.294786 2226 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:30.295465 2248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:30.295675 2226 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:30.295830 2226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "1c188491c2a140c38f1134be4a9050c6"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "585c90f55d57a66ed189b1434e7a9bd4"
server_key_iv: "86ba1c625c5037b5897101981a7989ac"
server_key_version: "encryptionkey@0"
I20260810 06:48:30.296196 2226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:30.313143 2226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:30.315467 2226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:30.315649 2226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:30.315913 2226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:30.316243 2226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:30.316303 2226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:30.316349 2226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:30.316373 2226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:30.322702 2226 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:36317
I20260810 06:48:30.322753 2378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:36317 every 8 connection(s)
I20260810 06:48:30.323125 2226 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:48:30.324667 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 2226
I20260810 06:48:30.325076 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:40573
--local_ip_for_outbound_sockets=127.15.232.195
--tserver_master_addrs=127.15.232.254:41517
--webserver_port=35083
--webserver_interface=127.15.232.195
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:48:30 dist-test-slave-vxj2 krb5kdc[1338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344510, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:48:30.334695 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.323979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:59695)
0810 06:48:30.324137 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:30.324180 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:30.324677 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:30.325090 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:30.325094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:30.325794 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:30.326005 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.326010 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.326991 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.326992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.327941 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.327944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.328474 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.328475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.328599 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.329283 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:30.329293 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:30.330811 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:30.332994 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:30.333001 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:30.333003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:30.333359 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:30.333622 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:30.333623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:30.333623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:30.333745 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:30.334107 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:30.334114 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:30.334251 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 06:48:30.334368 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:48:30.334511 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260810 06:48:30.334704 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.323958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:59695 (local address 127.15.232.254:41517)
0810 06:48:30.324021 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:30.324023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:30.324253 (+ 230us) server_negotiation.cc:407] Connection header received
0810 06:48:30.324927 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:30.324928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:30.324946 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:30.325031 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:30.326125 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.326855 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.328046 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.328372 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.328446 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.330933 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:30.330935 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:30.330935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:30.330946 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:30.332844 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:30.333446 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:30.333447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:30.333448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:30.333519 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:30.333838 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:30.333839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:30.333840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:30.334023 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:30.334104 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:30.334448 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 06:48:30.334563 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 06:48:30.335367 2379 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41517
I20260810 06:48:30.335497 2379 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:30.335755 2379 heartbeater.cc:511] Master 127.15.232.254:41517 requested a full tablet report, sending...
I20260810 06:48:30.336241 1998 ts_manager.cc:194] Registered new tserver with Master: 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317)
I20260810 06:48:30.336688 1998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:59695
W20260810 06:48:30.415455 2383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:30.415674 2383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:30.415701 2383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:30.416949 2383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:30.417001 2383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:30.417029 2383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:48:30.418556 2383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:40573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=35083
--tserver_master_addrs=127.15.232.254:41517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:48:30.418828 2383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:30.419111 2383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:30.424283 2389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:30.424280 2390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:30.424257 2392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:30.424502 2383 server_base.cc:1034] running on GCE node
I20260810 06:48:30.424779 2383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:30.425033 2383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:30.426196 2383 hybrid_clock.cc:648] HybridClock initialized: now 1786344510426188 us; error 38 us; skew 500 ppm
Aug 10 06:48:30 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344510, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:30.429025 2383 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:48:30.429500 2383 webserver.cc:460] Webserver started at http://127.15.232.195:35083/ using document root <none> and password file <none>
I20260810 06:48:30.429749 2383 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:30.429812 2383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:30.431187 2383 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:48:30.431864 2401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:30.432055 2383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:48:30.432130 2383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "af5e1eb6498741469fdb14401636c63b"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "a7a2122bae589af00b85cbb8d46bc89f"
server_key_iv: "0107f27512b96ccd98e91c54704c5c14"
server_key_version: "encryptionkey@0"
I20260810 06:48:30.432511 2383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:30.444393 2383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:30.446765 2383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:30.446940 2383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:30.447190 2383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:30.447536 2383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:30.447595 2383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:30.447638 2383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:30.447665 2383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:30.454226 2383 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:40573
I20260810 06:48:30.454275 2530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:40573 every 8 connection(s)
I20260810 06:48:30.454730 2383 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:48:30 dist-test-slave-vxj2 krb5kdc[1338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344510, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:48:30.461748 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 2383
I20260810 06:48:30.465679 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.455573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:50579 (local address 127.15.232.254:41517)
0810 06:48:30.455650 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:30.455652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:30.455898 (+ 246us) server_negotiation.cc:407] Connection header received
0810 06:48:30.456470 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:30.456472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:30.456486 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:30.456546 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:30.457670 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.458185 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.459267 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.459488 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.459574 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.462163 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:30.462165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:30.462166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:30.462176 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:30.464004 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:30.464530 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:30.464531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:30.464532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:30.464595 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:30.464945 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:30.464946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:30.464947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:30.465092 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:30.465198 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:30.465520 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 06:48:30.465592 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:48:30.465956 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.455553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:50579)
0810 06:48:30.455752 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:30.455792 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:30.456352 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:30.456628 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:30.456632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:30.457345 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:30.457557 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.457562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.458296 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.458297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.459162 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.459164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.459582 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.459583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.459723 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.460305 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:30.460314 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:30.462023 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:30.464089 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:30.464098 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:30.464100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:30.464433 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:30.464672 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:30.464673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:30.464674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:30.464836 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:30.465195 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:30.465202 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:30.465351 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 06:48:30.465493 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 06:48:30.465700 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260810 06:48:30.466555 2531 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41517
I20260810 06:48:30.466668 2531 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:30.466895 2531 heartbeater.cc:511] Master 127.15.232.254:41517 requested a full tablet report, sending...
I20260810 06:48:30.467309 1998 ts_manager.cc:194] Registered new tserver with Master: af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573)
I20260810 06:48:30.467689 1998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:50579
I20260810 06:48:30.469802 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.462230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34298 (local address 127.15.232.254:41517)
0810 06:48:30.462353 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:30.462356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:30.462485 (+ 129us) server_negotiation.cc:407] Connection header received
0810 06:48:30.462603 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:30.462604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:30.462623 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:30.462707 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:30.463331 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.463854 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.464945 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.465348 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.465431 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.467036 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:30.467038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:30.467039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:30.467046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:30.468572 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:30.468957 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:30.468958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:30.468958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:30.469023 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:30.469241 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:30.469243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:30.469243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:30.469382 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:30.469576 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:30.469614 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 06:48:30.469696 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260810 06:48:30.470633 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:48:30.473521 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.471014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34304 (local address 127.15.232.254:41517)
0810 06:48:30.471076 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:30.471078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:30.471178 (+ 100us) server_negotiation.cc:407] Connection header received
0810 06:48:30.471290 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:30.471291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:30.471340 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:30.471423 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:48:30.471562 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.472028 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.472801 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.473014 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.473090 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.473262 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:48:30.473312 (+ 50us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 06:48:30.473477 (+ 165us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34304: token expired
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 06:48:30.473598 2537 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34304: token expired
W20260810 06:48:30.473801 1924 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.232.254:41517)
I20260810 06:48:30.479594 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.474042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34316 (local address 127.15.232.254:41517)
0810 06:48:30.474082 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:30.474084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:30.474093 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:48:30.474118 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:30.474119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:30.474127 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:30.474168 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:30.474600 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.475072 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.475745 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.475941 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.475996 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.477139 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:30.477141 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:30.477141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:30.477147 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:30.478677 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:30.478999 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:30.478999 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:30.479000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:30.479053 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:30.479217 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:30.479217 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:30.479218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:30.479310 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:30.479382 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:30.479470 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 06:48:30.479530 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:48:30.480894 1998 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:48:30.482640 1998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:48:30.485142 2145 tablet_service.cc:1467] Processing CreateTablet for tablet 0b69870f952d4a54bb402592d46ee753 (DEFAULT_TABLE table=test-table [id=33eda9539aea4fb99ccfccf3252fbcd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:30.485435 2145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b69870f952d4a54bb402592d46ee753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:30.487305 2097 tablet_bootstrap.cc:492] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182: Bootstrap starting.
I20260810 06:48:30.487655 2097 tablet_bootstrap.cc:654] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:30.489212 2097 tablet_bootstrap.cc:492] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182: No bootstrap required, opened a new log
I20260810 06:48:30.489233 2539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.484715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:36317 (local address 127.0.0.1:60336)
0810 06:48:30.484882 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:30.484895 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:30.484954 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:30.485413 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:30.485414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:30.485416 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:30.485476 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.485478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.486921 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.486922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.488046 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.488049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.488913 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.488915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.488980 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.489005 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 06:48:30.489067 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 06:48:30.489122 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":71,"threads_started":1}
I20260810 06:48:30.489270 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.484956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60336 (local address 127.15.232.194:36317)
0810 06:48:30.485142 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:30.485191 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:30.485202 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:48:30.485256 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:30.485258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:30.485272 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:30.485376 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:30.485583 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.486809 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.488159 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.488818 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.488915 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.488977 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:30.489108 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 06:48:30.489181 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":108,"threads_started":1}
I20260810 06:48:30.489276 2097 ts_tablet_manager.cc:1397] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:48:30.489681 2097 raft_consensus.cc:348] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:30.489799 2097 raft_consensus.cc:374] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:30.489858 2097 raft_consensus.cc:729] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a989708ef3d54217a22640b3c6041182, State: Initialized, Role: FOLLOWER
I20260810 06:48:30.489912 2097 consensus_queue.cc:260] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:30.490023 2097 ts_tablet_manager.cc:1428] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 06:48:30.490131 2304 tablet_service.cc:1467] Processing CreateTablet for tablet 0b69870f952d4a54bb402592d46ee753 (DEFAULT_TABLE table=test-table [id=33eda9539aea4fb99ccfccf3252fbcd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:30.490504 2304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b69870f952d4a54bb402592d46ee753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:30.491098 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.484682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:40573 (local address 127.0.0.1:55286)
0810 06:48:30.484859 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:30.484875 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:30.484922 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:30.487550 (+ 2628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:30.487551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:30.487553 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:30.487604 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.487605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.488780 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.488781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.489826 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.489829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.490847 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.490848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.490932 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.490937 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:48:30.490972 (+ 35us) client_negotiation.cc:240] Negotiation successful
0810 06:48:30.491022 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260810 06:48:30.491506 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.484799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55286 (local address 127.15.232.195:40573)
0810 06:48:30.487375 (+ 2576us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:30.487380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:30.487392 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:48:30.487434 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:30.487437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:30.487455 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:30.487545 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:30.487671 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.488693 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.489930 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.490740 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.491280 (+ 540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.491356 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:30.491385 (+ 29us) server_negotiation.cc:299] Negotiation successful
0810 06:48:30.491427 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2530,"thread_start_us":130,"threads_started":1}
I20260810 06:48:30.492162 2454 tablet_service.cc:1467] Processing CreateTablet for tablet 0b69870f952d4a54bb402592d46ee753 (DEFAULT_TABLE table=test-table [id=33eda9539aea4fb99ccfccf3252fbcd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:30.492542 2454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b69870f952d4a54bb402592d46ee753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:30.495734 2543 tablet_bootstrap.cc:492] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6: Bootstrap starting.
I20260810 06:48:30.495704 2544 tablet_bootstrap.cc:492] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b: Bootstrap starting.
I20260810 06:48:30.496335 2543 tablet_bootstrap.cc:654] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:30.496376 2544 tablet_bootstrap.cc:654] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:30.496860 2543 log.cc:824] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:30.496912 2544 log.cc:824] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:30.497785 2544 tablet_bootstrap.cc:492] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b: No bootstrap required, opened a new log
I20260810 06:48:30.497783 2543 tablet_bootstrap.cc:492] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6: No bootstrap required, opened a new log
I20260810 06:48:30.497848 2544 ts_tablet_manager.cc:1397] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:48:30.497886 2543 ts_tablet_manager.cc:1397] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:48:30.499396 2543 raft_consensus.cc:348] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:30.499625 2543 raft_consensus.cc:374] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:30.499543 2544 raft_consensus.cc:348] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:30.499673 2543 raft_consensus.cc:729] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c188491c2a140c38f1134be4a9050c6, State: Initialized, Role: FOLLOWER
I20260810 06:48:30.499719 2544 raft_consensus.cc:374] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:30.499763 2544 raft_consensus.cc:729] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af5e1eb6498741469fdb14401636c63b, State: Initialized, Role: FOLLOWER
I20260810 06:48:30.499809 2543 consensus_queue.cc:260] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:30.499893 2544 consensus_queue.cc:260] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:30.500130 2379 heartbeater.cc:503] Master 127.15.232.254:41517 was elected leader, sending a full tablet report...
I20260810 06:48:30.500145 2543 ts_tablet_manager.cc:1428] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:48:30.500207 2531 heartbeater.cc:503] Master 127.15.232.254:41517 was elected leader, sending a full tablet report...
I20260810 06:48:30.500468 2544 ts_tablet_manager.cc:1428] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260810 06:48:30.573732 2380 tablet.cc:2369] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:48:30.682538 2222 tablet.cc:2369] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:30.687526 2547 raft_consensus.cc:482] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:30.687659 2547 raft_consensus.cc:504] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:30.688113 2547 leader_election.cc:288] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573), a989708ef3d54217a22640b3c6041182 (127.15.232.193:36535)
I20260810 06:48:30.691325 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.688494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:36275 (local address 127.15.232.195:40573)
0810 06:48:30.688540 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:30.688542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:30.688741 (+ 199us) server_negotiation.cc:407] Connection header received
0810 06:48:30.688775 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:30.688776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:30.688843 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:30.688897 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:30.689084 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.689579 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.690548 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.690947 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.691038 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.691048 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:30.691200 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 06:48:30.691240 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:48:30.691360 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.688355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:40573 (local address 127.15.232.194:36275)
0810 06:48:30.688629 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:30.688640 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:30.688699 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:30.688931 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:30.688932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:30.688934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:30.689001 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.689003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.689679 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.689679 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.690442 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.690444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.691038 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.691040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.691119 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.691123 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:48:30.691194 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 06:48:30.691235 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":79,"thread_start_us":123,"threads_started":1}
I20260810 06:48:30.691350 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.688432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:36177 (local address 127.15.232.193:36535)
0810 06:48:30.688490 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:30.688492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:30.688506 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:48:30.688542 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:30.688543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:30.688572 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:30.688663 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:30.688865 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.689556 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.690521 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.690981 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.691086 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.691166 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:30.691234 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 06:48:30.691271 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:48:30.691360 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.688333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:36535 (local address 127.15.232.194:36177)
0810 06:48:30.688399 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:30.688412 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:30.688466 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:30.688697 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:30.688698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:30.688700 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:30.688780 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.688782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.689645 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.689646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.690417 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:30.690419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.691086 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.691087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:30.691164 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.691168 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:48:30.691227 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 06:48:30.691276 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:48:30.691727 2169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b69870f952d4a54bb402592d46ee753" candidate_uuid: "1c188491c2a140c38f1134be4a9050c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a989708ef3d54217a22640b3c6041182" is_pre_election: true
I20260810 06:48:30.691771 2477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b69870f952d4a54bb402592d46ee753" candidate_uuid: "1c188491c2a140c38f1134be4a9050c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af5e1eb6498741469fdb14401636c63b" is_pre_election: true
I20260810 06:48:30.691917 2169 raft_consensus.cc:2454] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c188491c2a140c38f1134be4a9050c6 in term 0.
I20260810 06:48:30.691934 2477 raft_consensus.cc:2454] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c188491c2a140c38f1134be4a9050c6 in term 0.
I20260810 06:48:30.692144 2252 leader_election.cc:302] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c188491c2a140c38f1134be4a9050c6, a989708ef3d54217a22640b3c6041182; no voters:
I20260810 06:48:30.692281 2547 raft_consensus.cc:2781] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:48:30.692339 2547 raft_consensus.cc:482] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:48:30.692363 2547 raft_consensus.cc:3037] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:30.693080 2547 raft_consensus.cc:504] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:30.693272 2547 leader_election.cc:288] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [CANDIDATE]: Term 1 election: Requested vote from peers af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573), a989708ef3d54217a22640b3c6041182 (127.15.232.193:36535)
I20260810 06:48:30.693485 2169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b69870f952d4a54bb402592d46ee753" candidate_uuid: "1c188491c2a140c38f1134be4a9050c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a989708ef3d54217a22640b3c6041182"
I20260810 06:48:30.693581 2169 raft_consensus.cc:3037] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:30.694398 2169 raft_consensus.cc:2454] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c188491c2a140c38f1134be4a9050c6 in term 1.
I20260810 06:48:30.694540 2477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b69870f952d4a54bb402592d46ee753" candidate_uuid: "1c188491c2a140c38f1134be4a9050c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af5e1eb6498741469fdb14401636c63b"
I20260810 06:48:30.694595 2252 leader_election.cc:302] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c188491c2a140c38f1134be4a9050c6, a989708ef3d54217a22640b3c6041182; no voters:
I20260810 06:48:30.694641 2477 raft_consensus.cc:3037] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:30.694769 2547 raft_consensus.cc:2781] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:30.694970 2547 raft_consensus.cc:686] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 1 LEADER]: Becoming Leader. State: Replica: 1c188491c2a140c38f1134be4a9050c6, State: Running, Role: LEADER
I20260810 06:48:30.695087 2547 consensus_queue.cc:237] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:30.695592 2477 raft_consensus.cc:2454] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c188491c2a140c38f1134be4a9050c6 in term 1.
I20260810 06:48:30.696089 1998 catalog_manager.cc:5495] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194). New cstate: current_term: 1 leader_uuid: "1c188491c2a140c38f1134be4a9050c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } health_report { overall_health: UNKNOWN } } }
W20260810 06:48:30.705375 2532 tablet.cc:2369] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:30.718204 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.715646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60352 (local address 127.15.232.194:36317)
0810 06:48:30.715732 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:30.715737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:30.715751 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:48:30.715808 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:30.715809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:30.715838 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:30.715910 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:48:30.716037 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.716525 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.717368 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:30.717582 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:30.717642 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:30.717840 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:48:30.717949 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:48:30.718013 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:30.718045 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 06:48:30.718065 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:48:30.720407 2169 raft_consensus.cc:1261] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Refusing update from remote peer 1c188491c2a140c38f1134be4a9050c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:30.720407 2477 raft_consensus.cc:1261] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b [term 1 FOLLOWER]: Refusing update from remote peer 1c188491c2a140c38f1134be4a9050c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:30.720918 2547 consensus_queue.cc:1035] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:30.721093 2550 consensus_queue.cc:1035] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:30.723886 2556 mvcc.cc:205] Tried to move back new op lower bound from 7316867115906879488 to 7316867115807764480. Current Snapshot: MvccSnapshot[applied={T|T < 7316867115906879488}]
I20260810 06:48:30.724067 2554 mvcc.cc:205] Tried to move back new op lower bound from 7316867115906879488 to 7316867115807764480. Current Snapshot: MvccSnapshot[applied={T|T < 7316867115906879488}]
I20260810 06:48:30.725876 1998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:30.725972 1998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:30.726756 1998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Sending DeleteTablet for 3 replicas of tablet 0b69870f952d4a54bb402592d46ee753
I20260810 06:48:30.727089 2304 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b69870f952d4a54bb402592d46ee753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:30 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:60336
I20260810 06:48:30.727089 2454 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b69870f952d4a54bb402592d46ee753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:30 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:55286
I20260810 06:48:30.727137 2145 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b69870f952d4a54bb402592d46ee753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:30 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:41436
I20260810 06:48:30.727365 2564 tablet_replica.cc:330] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6: stopping tablet replica
I20260810 06:48:30.727396 2563 tablet_replica.cc:330] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b: stopping tablet replica
I20260810 06:48:30.727403 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 2046
I20260810 06:48:30.730185 2563 raft_consensus.cc:2229] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:30.730346 2563 raft_consensus.cc:2258] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:30.730607 2564 raft_consensus.cc:2229] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:48:30.730779 2564 raft_consensus.cc:2258] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:30.730942 2563 ts_tablet_manager.cc:1905] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:30.731346 2564 ts_tablet_manager.cc:1905] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:30.733480 2563 ts_tablet_manager.cc:1918] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:30.733569 2563 log.cc:1196] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/0b69870f952d4a54bb402592d46ee753
I20260810 06:48:30.733817 2564 ts_tablet_manager.cc:1918] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:30.733899 2563 ts_tablet_manager.cc:1939] T 0b69870f952d4a54bb402592d46ee753 P af5e1eb6498741469fdb14401636c63b: Deleting consensus metadata
I20260810 06:48:30.734002 2564 log.cc:1196] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/0b69870f952d4a54bb402592d46ee753
I20260810 06:48:30.734370 2564 ts_tablet_manager.cc:1939] T 0b69870f952d4a54bb402592d46ee753 P 1c188491c2a140c38f1134be4a9050c6: Deleting consensus metadata
I20260810 06:48:30.734400 1982 catalog_manager.cc:4841] TS af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573): tablet 0b69870f952d4a54bb402592d46ee753 (table test-table [id=33eda9539aea4fb99ccfccf3252fbcd4]) successfully deleted
I20260810 06:48:30.734844 1983 catalog_manager.cc:4841] TS 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317): tablet 0b69870f952d4a54bb402592d46ee753 (table test-table [id=33eda9539aea4fb99ccfccf3252fbcd4]) successfully deleted
W20260810 06:48:30.737679 1985 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:36535 (error 108)
I20260810 06:48:30.738010 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 2226
I20260810 06:48:30.739784 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:30.738174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:36535 (local address 127.0.0.1:41444)
0810 06:48:30.739690 (+ 1516us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:30.739743 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:36535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1492}
W20260810 06:48:30.739941 1985 catalog_manager.cc:4584] TS a989708ef3d54217a22640b3c6041182 (127.15.232.193:36535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0b69870f952d4a54bb402592d46ee753: Network error: Client connection negotiation failed: client connection to 127.15.232.193:36535: connect: Connection refused (error 111)
I20260810 06:48:30.744773 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 2383
I20260810 06:48:30.755390 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 1964
I20260810 06:48:50.763972 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:41517
--webserver_interface=127.15.232.254
--webserver_port=42287
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:41517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:50.850268 2569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:50.850483 2569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:50.850507 2569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:50.851665 2569 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:48:50.851712 2569 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:48:50.851725 2569 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:48:50.851737 2569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:50.851750 2569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:50.851763 2569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:50.851776 2569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:50.851788 2569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:50.853521 2569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:41517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:41517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=42287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:48:50.853750 2569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:50.853940 2569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:50.859249 2575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:50.859229 2578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:50.859262 2569 server_base.cc:1034] running on GCE node
W20260810 06:48:50.859210 2574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:50.859705 2569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:50.860095 2569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:50.861227 2569 hybrid_clock.cc:648] HybridClock initialized: now 1786344530861207 us; error 51 us; skew 500 ppm
Aug 10 06:48:50 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344530, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:50.864256 2569 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:48:50.864715 2569 webserver.cc:460] Webserver started at http://127.15.232.254:42287/ using document root <none> and password file <none>
I20260810 06:48:50.864939 2569 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:50.864988 2569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:50.866515 2569 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:50.867285 2585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:50.867506 2569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:50.867604 2569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "550492d95fcf403ba087a133e7b832c8"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "bdcfb4ffb640c47acfe6cb19d645d416"
server_key_iv: "a29e7fa1ab535db911a4f2ae89069d89"
server_key_version: "encryptionkey@0"
I20260810 06:48:50.867990 2569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:50.884558 2569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:50.887079 2569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:50.887202 2569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:50.891206 2569 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:41517
I20260810 06:48:50.891407 2647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:41517 every 8 connection(s)
I20260810 06:48:50.891682 2569 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:50.893481 2648 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:48:50.894204 2648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Bootstrap starting.
I20260810 06:48:50.899842 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 2569
I20260810 06:48:50.900188 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:36535
--local_ip_for_outbound_sockets=127.15.232.193
--tserver_master_addrs=127.15.232.254:41517
--webserver_port=33475
--webserver_interface=127.15.232.193
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:48:50.903455 2648 log.cc:824] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:50.906199 2648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:48:50.906633 2648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Bootstrap complete.
I20260810 06:48:50.909119 2648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } }
I20260810 06:48:50.909494 2648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550492d95fcf403ba087a133e7b832c8, State: Initialized, Role: FOLLOWER
I20260810 06:48:50.909689 2648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } }
I20260810 06:48:50.909778 2648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:50.909839 2648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:50.909899 2648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:48:50.911021 2648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } }
I20260810 06:48:50.911159 2648 leader_election.cc:302] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 550492d95fcf403ba087a133e7b832c8; no voters:
I20260810 06:48:50.911401 2648 leader_election.cc:288] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 06:48:50.911442 2653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:48:50.911576 2653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [term 3 LEADER]: Becoming Leader. State: Replica: 550492d95fcf403ba087a133e7b832c8, State: Running, Role: LEADER
I20260810 06:48:50.911679 2653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } }
I20260810 06:48:50.911712 2648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:50.912369 2655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 550492d95fcf403ba087a133e7b832c8. Latest consensus state: current_term: 3 leader_uuid: "550492d95fcf403ba087a133e7b832c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } } }
I20260810 06:48:50.912457 2655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:50.912633 2654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "550492d95fcf403ba087a133e7b832c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550492d95fcf403ba087a133e7b832c8" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41517 } } }
I20260810 06:48:50.912701 2654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:50.913105 2664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:50.914184 2664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:50.914330 2664 catalog_manager.cc:1259] Loaded cluster ID: 6416e738be654f2f847fec6ce4b9f0ff
I20260810 06:48:50.914569 2664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:50.915886 2664 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:50.916095 2664 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Loaded TSK: 0
I20260810 06:48:50.916152 2664 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 550492d95fcf403ba087a133e7b832c8: Loaded TSK: 1
F20260810 06:48:50.916425 2664 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f156c2159ad google::LogMessage::Fail() at ??:0
@ 0x7f156c21888a google::LogMessage::SendToLog() at ??:0
@ 0x7f156c2154de google::LogMessage::Flush() at ??:0
@ 0x7f156c216a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f156e982104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f156e97fd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f157416f1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f157416f48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f157416f594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f1574135c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f157413fad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f156e0e30d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f156e0d8954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f156ef3c6db start_thread at ??:0
@ 0x7f156ad1361f clone at ??:0
W20260810 06:48:51.000972 2650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:51.001274 2650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:51.001308 2650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:51.002548 2650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:51.002596 2650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:51.002624 2650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:48:51.004194 2650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:36535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=33475
--tserver_master_addrs=127.15.232.254:41517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:48:51.004474 2650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:51.004734 2650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:51.010198 2698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:51.010212 2697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:51.010192 2701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:51.010857 2650 server_base.cc:1034] running on GCE node
I20260810 06:48:51.011129 2650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:51.011456 2650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:51.012650 2650 hybrid_clock.cc:648] HybridClock initialized: now 1786344531012634 us; error 29 us; skew 500 ppm
Aug 10 06:48:51 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344531, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:51.016147 2650 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:48:51.016727 2650 webserver.cc:460] Webserver started at http://127.15.232.193:33475/ using document root <none> and password file <none>
I20260810 06:48:51.017007 2650 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:51.017086 2650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:51.018692 2650 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:51.019724 2709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:51.019973 2650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:51.020092 2650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "a989708ef3d54217a22640b3c6041182"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "53623c9ab52f89312a10822169dce6a9"
server_key_iv: "09567922ae2e4118202cf4534e773046"
server_key_version: "encryptionkey@0"
I20260810 06:48:51.020489 2650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:51.052493 2650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:51.055119 2650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:51.055315 2650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:51.055591 2650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:51.056175 2717 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 06:48:51.058534 2650 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 06:48:51.058611 2650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260810 06:48:51.058662 2650 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 06:48:51.059432 2650 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 06:48:51.059501 2650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 06:48:51.059594 2717 tablet_bootstrap.cc:492] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182: Bootstrap starting.
I20260810 06:48:51.067399 2650 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:36535
I20260810 06:48:51.067585 2837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:36535 every 8 connection(s)
I20260810 06:48:51.067893 2650 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:48:51.072871 2717 log.cc:824] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:51.076301 2717 tablet_bootstrap.cc:492] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:48:51.076478 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 2650
I20260810 06:48:51.076756 2717 tablet_bootstrap.cc:492] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182: Bootstrap complete.
I20260810 06:48:51.076790 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:36317
--local_ip_for_outbound_sockets=127.15.232.194
--tserver_master_addrs=127.15.232.254:41517
--webserver_port=34579
--webserver_interface=127.15.232.194
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:48:51.078224 2717 ts_tablet_manager.cc:1397] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.010s
I20260810 06:48:51.079921 2717 raft_consensus.cc:348] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:51.080296 2717 raft_consensus.cc:729] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a989708ef3d54217a22640b3c6041182, State: Initialized, Role: FOLLOWER
I20260810 06:48:51.080497 2717 consensus_queue.cc:260] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:51.080844 2717 ts_tablet_manager.cc:1428] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
W20260810 06:48:51.149529 2840 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:48:51.068814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:39679)
0810 06:48:51.069043 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:51.069087 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:51.069672 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:51.149341 (+ 79669us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":115,"threads_started":1}
W20260810 06:48:51.149811 2838 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:41517 (0 consecutive failures): Network error: Failed to ping master at 127.15.232.254:41517: Client connection negotiation failed: client connection to 127.15.232.254:41517: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:48:51.150197 2840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:51.150059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:49969)
0810 06:48:51.150123 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:51.150174 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 06:48:51.150571 2840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:51.150475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:59273)
0810 06:48:51.150525 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:51.150549 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:48:51.150689 2838 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:48:51.166957 2843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:51.167218 2843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:51.167263 2843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:51.168512 2843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:51.168586 2843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:51.168633 2843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:48:51.170334 2843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:36317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=34579
--tserver_master_addrs=127.15.232.254:41517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:48:51.170626 2843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:51.170892 2843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:51.176422 2850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:51.176422 2849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:51.176469 2853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:51.176748 2843 server_base.cc:1034] running on GCE node
I20260810 06:48:51.176986 2843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:51.177294 2843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:51.178459 2843 hybrid_clock.cc:648] HybridClock initialized: now 1786344531178444 us; error 45 us; skew 500 ppm
Aug 10 06:48:51 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344531, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:51.181222 2843 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:48:51.181740 2843 webserver.cc:460] Webserver started at http://127.15.232.194:34579/ using document root <none> and password file <none>
I20260810 06:48:51.181982 2843 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:51.182053 2843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:51.183553 2843 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:48:51.184202 2861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:51.184422 2843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:51.184535 2843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "1c188491c2a140c38f1134be4a9050c6"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "585c90f55d57a66ed189b1434e7a9bd4"
server_key_iv: "86ba1c625c5037b5897101981a7989ac"
server_key_version: "encryptionkey@0"
I20260810 06:48:51.184911 2843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:51.197103 2843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:51.199674 2843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:51.199821 2843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:51.200103 2843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:51.200444 2843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:51.200479 2843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:51.200501 2843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:51.200547 2843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:51.206826 2843 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:36317
I20260810 06:48:51.207003 2994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:36317 every 8 connection(s)
I20260810 06:48:51.207309 2843 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:48:51.208456 2998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:51.208021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:58165)
0810 06:48:51.208190 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:51.208299 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
W20260810 06:48:51.208904 2995 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:41517 (0 consecutive failures): Network error: Failed to ping master at 127.15.232.254:41517: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
I20260810 06:48:51.209254 2998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:51.209131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:51333)
0810 06:48:51.209201 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:51.209229 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 06:48:51.209544 2998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:51.209459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:45961)
0810 06:48:51.209503 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:51.209525 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 06:48:51.209657 2995 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:48:51.215569 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 2843
I20260810 06:48:51.215832 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:40573
--local_ip_for_outbound_sockets=127.15.232.195
--tserver_master_addrs=127.15.232.254:41517
--webserver_port=35083
--webserver_interface=127.15.232.195
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:51.301981 2999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:51.302253 2999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:51.302302 2999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:51.303516 2999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:51.303589 2999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:51.303637 2999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:48:51.305235 2999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:40573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=35083
--tserver_master_addrs=127.15.232.254:41517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:48:51.305532 2999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:51.305779 2999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:51.311228 3005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:51.311250 3008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:51.311309 3006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:51.311297 2999 server_base.cc:1034] running on GCE node
I20260810 06:48:51.311709 2999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:51.312021 2999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:51.313259 2999 hybrid_clock.cc:648] HybridClock initialized: now 1786344531313225 us; error 44 us; skew 500 ppm
Aug 10 06:48:51 dist-test-slave-vxj2 krb5kdc[1338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344531, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:51.316283 2999 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:48:51.316804 2999 webserver.cc:460] Webserver started at http://127.15.232.195:35083/ using document root <none> and password file <none>
I20260810 06:48:51.317034 2999 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:51.317082 2999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:51.318534 2999 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:51.319162 3016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:51.319332 2999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:48:51.319427 2999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "af5e1eb6498741469fdb14401636c63b"
format_stamp: "Formatted at 2026-08-10 06:48:09 on dist-test-slave-vxj2"
server_key: "a7a2122bae589af00b85cbb8d46bc89f"
server_key_iv: "0107f27512b96ccd98e91c54704c5c14"
server_key_version: "encryptionkey@0"
I20260810 06:48:51.319772 2999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:51.328627 2999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:51.331046 2999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:51.331197 2999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:51.331446 2999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:51.331712 2999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:51.331743 2999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:51.331764 2999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:51.331777 2999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:51.337728 2999 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:40573
I20260810 06:48:51.337811 3151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:40573 every 8 connection(s)
I20260810 06:48:51.338171 2999 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:48:51.339267 3155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:51.338891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:59729)
0810 06:48:51.339048 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:51.339122 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
W20260810 06:48:51.339704 3152 heartbeater.cc:644] Failed to heartbeat to 127.15.232.254:41517 (0 consecutive failures): Network error: Failed to ping master at 127.15.232.254:41517: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
I20260810 06:48:51.340039 3155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:51.339933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:58209)
0810 06:48:51.339991 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:51.340019 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:48:51.340305 3155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:51.340255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:36343)
0810 06:48:51.340275 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:51.340294 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260810 06:48:51.340431 3152 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:48:51.341517 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 2999
I20260810 06:48:52.151664 3159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:52.151279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:42077)
0810 06:48:52.151475 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:52.151571 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":81,"threads_started":1}
I20260810 06:48:52.210355 3160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:52.210080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:46543)
0810 06:48:52.210230 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:52.210280 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":63,"threads_started":1}
I20260810 06:48:52.341109 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:52.340859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:50687)
0810 06:48:52.341009 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:52.341054 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260810 06:48:52.781070 3162 raft_consensus.cc:482] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:52.781313 3162 raft_consensus.cc:504] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:52.781744 3162 leader_election.cc:288] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573), 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317)
Aug 10 06:48:52 dist-test-slave-vxj2 krb5kdc[1338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344531, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.194@KRBTEST.COM
Aug 10 06:48:52 dist-test-slave-vxj2 krb5kdc[1338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344531, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.195@KRBTEST.COM
I20260810 06:48:52.800020 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:52.782080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:47661 (local address 127.15.232.194:36317)
0810 06:48:52.782336 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:52.782344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:52.782365 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:48:52.782816 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:52.782818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:52.782862 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:52.782983 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:52.783896 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:52.784922 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:52.786078 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:52.786522 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:52.786640 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:52.794532 (+ 7892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:52.794538 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:52.794539 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:52.794552 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:52.797225 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:52.798047 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:52.798052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:52.798052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:52.798197 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:52.798650 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:52.798651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:52.798651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:52.799137 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:52.799239 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:52.799711 (+ 472us) server_negotiation.cc:299] Negotiation successful
0810 06:48:52.799917 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":56,"threads_started":1}
I20260810 06:48:52.800010 3167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:52.782122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:54203 (local address 127.15.232.195:40573)
0810 06:48:52.782320 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:52.782334 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:52.782373 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:48:52.782850 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:52.782859 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:52.782900 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:52.783076 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:52.783849 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:52.784923 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:52.786071 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:52.786525 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:52.786662 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:52.794866 (+ 8204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:52.794872 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:52.794872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:52.794888 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:52.797208 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:52.798047 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:52.798052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:52.798052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:52.798198 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:52.798650 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:52.798651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:52.798651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:52.799137 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:52.799263 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:52.799711 (+ 448us) server_negotiation.cc:299] Negotiation successful
0810 06:48:52.799891 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":59,"threads_started":1}
I20260810 06:48:52.799991 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:52.782005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:40573 (local address 127.15.232.193:54203)
0810 06:48:52.782140 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:52.782153 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:52.782219 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:52.783085 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:52.783088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:52.783493 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:52.783704 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:52.783710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:52.785044 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:52.785045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:52.785969 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:52.785971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:52.786642 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:52.786643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:52.786756 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:52.792384 (+ 5628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:52.792397 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:52.794677 (+ 2280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:52.797353 (+ 2676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:52.797371 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:52.797374 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:52.797917 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:52.798299 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:52.798300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:52.798300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:52.798542 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:52.799255 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:52.799265 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:52.799458 (+ 193us) client_negotiation.cc:769] Sending connection context
0810 06:48:52.799595 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 06:48:52.799840 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":56,"thread_start_us":58,"threads_started":1}
I20260810 06:48:52.799993 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:52.781957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:36317 (local address 127.15.232.193:47661)
0810 06:48:52.782122 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:52.782134 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:52.782196 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:52.783083 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:52.783086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:52.783492 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:52.783704 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:52.783710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:52.785064 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:52.785065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:52.785970 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:52.785973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:52.786624 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:52.786625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:52.786758 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:52.792384 (+ 5626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:52.792397 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:52.794301 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:52.797353 (+ 3052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:52.797371 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:52.797374 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:52.797896 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:52.798299 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:52.798300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:52.798300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:52.798542 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:52.799255 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:52.799266 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:52.799458 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 06:48:52.799596 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 06:48:52.799899 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":188,"thread_start_us":92,"threads_started":1}
W20260810 06:48:52.801052 2711 leader_election.cc:341] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573): Not found: Tablet not found: 0b69870f952d4a54bb402592d46ee753
W20260810 06:48:52.801700 2712 leader_election.cc:341] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317): Not found: Tablet not found: 0b69870f952d4a54bb402592d46ee753
I20260810 06:48:52.801761 2712 leader_election.cc:302] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a989708ef3d54217a22640b3c6041182; no voters: 1c188491c2a140c38f1134be4a9050c6, af5e1eb6498741469fdb14401636c63b
I20260810 06:48:52.801919 3162 raft_consensus.cc:2726] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:48:53.152457 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:53.152290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:60067)
0810 06:48:53.152367 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:53.152414 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 06:48:53.211231 3168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:53.210923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:58249)
0810 06:48:53.211115 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:53.211175 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":78,"threads_started":1}
I20260810 06:48:53.341894 3169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:53.341640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:35335)
0810 06:48:53.341805 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:53.341851 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
I20260810 06:48:54.153354 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:54.153097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:34525)
0810 06:48:54.153266 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:54.153306 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":101,"threads_started":1}
I20260810 06:48:54.212010 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:54.211783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:44261)
0810 06:48:54.211916 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:54.211963 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260810 06:48:54.306145 3173 raft_consensus.cc:482] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:54.306257 3173 raft_consensus.cc:504] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:54.306502 3173 leader_election.cc:288] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573), 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317)
W20260810 06:48:54.307003 2711 leader_election.cc:341] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573): Not found: Tablet not found: 0b69870f952d4a54bb402592d46ee753
W20260810 06:48:54.307096 2712 leader_election.cc:341] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317): Not found: Tablet not found: 0b69870f952d4a54bb402592d46ee753
I20260810 06:48:54.307206 2712 leader_election.cc:302] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a989708ef3d54217a22640b3c6041182; no voters: 1c188491c2a140c38f1134be4a9050c6, af5e1eb6498741469fdb14401636c63b
I20260810 06:48:54.307309 3173 raft_consensus.cc:2726] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:48:54.342737 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:54.342451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:35741)
0810 06:48:54.342634 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:54.342679 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":86,"threads_started":1}
I20260810 06:48:55.154100 3175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:55.153876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:40455)
0810 06:48:55.154011 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:55.154048 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260810 06:48:55.212854 3176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:55.212601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:41995)
0810 06:48:55.212755 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:55.212803 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":62,"threads_started":1}
I20260810 06:48:55.343597 3177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:55.343342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:55745)
0810 06:48:55.343505 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:55.343552 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":78,"threads_started":1}
I20260810 06:48:56.154964 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:56.154680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:42477)
0810 06:48:56.154871 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:56.154914 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":67,"threads_started":1}
I20260810 06:48:56.213712 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:56.213470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:35727)
0810 06:48:56.213619 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:56.213665 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
I20260810 06:48:56.344507 3181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:56.344244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:34283)
0810 06:48:56.344403 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:56.344456 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260810 06:48:56.899506 3182 raft_consensus.cc:482] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:56.899645 3182 raft_consensus.cc:504] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:48:56.900063 3182 leader_election.cc:288] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573), 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317)
W20260810 06:48:56.900465 2711 leader_election.cc:341] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573): Not found: Tablet not found: 0b69870f952d4a54bb402592d46ee753
W20260810 06:48:56.900537 2712 leader_election.cc:341] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317): Not found: Tablet not found: 0b69870f952d4a54bb402592d46ee753
I20260810 06:48:56.900571 2712 leader_election.cc:302] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a989708ef3d54217a22640b3c6041182; no voters: 1c188491c2a140c38f1134be4a9050c6, af5e1eb6498741469fdb14401636c63b
I20260810 06:48:56.900650 3182 raft_consensus.cc:2726] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:48:57.155845 3184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:57.155583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:52145)
0810 06:48:57.155765 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:57.155796 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":106,"threads_started":1}
I20260810 06:48:57.214494 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:57.214239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:52073)
0810 06:48:57.214402 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:57.214447 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260810 06:48:57.345359 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:57.345114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:44877)
0810 06:48:57.345265 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:57.345315 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260810 06:48:58.156780 3188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:58.156530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:37467)
0810 06:48:58.156686 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:58.156725 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260810 06:48:58.215266 3189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:58.215030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:59811)
0810 06:48:58.215173 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:58.215217 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260810 06:48:58.346127 3190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:58.345903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:42687)
0810 06:48:58.346040 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:58.346083 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260810 06:48:59.157575 3191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:59.157307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:39865)
0810 06:48:59.157485 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:59.157526 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260810 06:48:59.216235 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:59.215964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:42431)
0810 06:48:59.216116 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:59.216159 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260810 06:48:59.346973 3193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:59.346729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:55633)
0810 06:48:59.346882 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:59.346928 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260810 06:49:00.158507 3194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:00.158237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:39321)
0810 06:49:00.158406 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:00.158456 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":75,"threads_started":1}
I20260810 06:49:00.217075 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:00.216820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:55303)
0810 06:49:00.216984 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:00.217028 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
I20260810 06:49:00.347689 3196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:00.347462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:59601)
0810 06:49:00.347606 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:00.347641 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260810 06:49:01.159439 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:01.159179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:38587)
0810 06:49:01.159334 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:01.159389 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260810 06:49:01.217907 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:01.217619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:41095)
0810 06:49:01.217802 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:01.217858 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":163,"threads_started":1}
I20260810 06:49:01.348500 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:01.348247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:45053)
0810 06:49:01.348399 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:01.348448 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260810 06:49:01.410873 3200 raft_consensus.cc:482] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:01.410995 3200 raft_consensus.cc:504] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:49:01.411188 3200 leader_election.cc:288] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573), 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317)
W20260810 06:49:01.411653 2712 leader_election.cc:341] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317): Not found: Tablet not found: 0b69870f952d4a54bb402592d46ee753
W20260810 06:49:01.411787 2711 leader_election.cc:341] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573): Not found: Tablet not found: 0b69870f952d4a54bb402592d46ee753
I20260810 06:49:01.411849 2711 leader_election.cc:302] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a989708ef3d54217a22640b3c6041182; no voters: 1c188491c2a140c38f1134be4a9050c6, af5e1eb6498741469fdb14401636c63b
I20260810 06:49:01.411968 3200 raft_consensus.cc:2726] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:49:02.160305 3201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:02.160035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:60365)
0810 06:49:02.160190 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:02.160240 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260810 06:49:02.218773 3202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:02.218472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:55375)
0810 06:49:02.218660 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:02.218712 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
I20260810 06:49:02.349375 3203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:02.349060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:49719)
0810 06:49:02.349280 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:02.349327 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":77,"threads_started":1}
I20260810 06:49:03.161396 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:03.161050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:52717)
0810 06:49:03.161291 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:03.161346 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":152,"threads_started":1}
I20260810 06:49:03.219643 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:03.219381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:45705)
0810 06:49:03.219546 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:03.219596 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":75,"threads_started":1}
I20260810 06:49:03.350145 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:03.349888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:48597)
0810 06:49:03.350058 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:03.350099 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260810 06:49:03.925937 3208 raft_consensus.cc:482] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:03.926036 3208 raft_consensus.cc:504] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5e1eb6498741469fdb14401636c63b" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 40573 } } peers { permanent_uuid: "1c188491c2a140c38f1134be4a9050c6" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36317 } } peers { permanent_uuid: "a989708ef3d54217a22640b3c6041182" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36535 } }
I20260810 06:49:03.926246 3208 leader_election.cc:288] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573), 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317)
W20260810 06:49:03.926712 2712 leader_election.cc:341] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c188491c2a140c38f1134be4a9050c6 (127.15.232.194:36317): Not found: Tablet not found: 0b69870f952d4a54bb402592d46ee753
W20260810 06:49:03.926792 2711 leader_election.cc:341] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af5e1eb6498741469fdb14401636c63b (127.15.232.195:40573): Not found: Tablet not found: 0b69870f952d4a54bb402592d46ee753
I20260810 06:49:03.926828 2712 leader_election.cc:302] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a989708ef3d54217a22640b3c6041182; no voters: 1c188491c2a140c38f1134be4a9050c6, af5e1eb6498741469fdb14401636c63b
I20260810 06:49:03.926918 3208 raft_consensus.cc:2726] T 0b69870f952d4a54bb402592d46ee753 P a989708ef3d54217a22640b3c6041182 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:49:04.162446 3210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:04.162176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:60441)
0810 06:49:04.162341 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:04.162393 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":71,"threads_started":1}
I20260810 06:49:04.220459 3211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:04.220196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:35903)
0810 06:49:04.220368 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:04.220408 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260810 06:49:04.350852 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:04.350634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:50041)
0810 06:49:04.350767 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:04.350807 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260810 06:49:05.163355 3213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:05.163066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:34959)
0810 06:49:05.163250 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:05.163304 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":72,"threads_started":1}
I20260810 06:49:05.221499 3214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:05.221091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:49037)
0810 06:49:05.221390 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:05.221450 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":248,"threads_started":1}
I20260810 06:49:05.351678 3215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:05.351431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:49929)
0810 06:49:05.351580 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:05.351628 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260810 06:49:06.164357 3216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:06.164063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.193:59589)
0810 06:49:06.164251 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:06.164303 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":88,"threads_started":1}
I20260810 06:49:06.222420 3218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:06.222123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.194:59727)
0810 06:49:06.222322 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:06.222372 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260810 06:49:06.350769 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 2650
I20260810 06:49:06.354686 3219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:06.354467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41517 (local address 127.15.232.195:34679)
0810 06:49:06.354598 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:06.354639 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.254:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260810 06:49:06.358376 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 2843
I20260810 06:49:06.365605 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 2999
2026-08-10T06:49:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58582 ms)
[----------] 30 tests from SecurityITest (242394 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:06 dist-test-slave-vxj2 krb5kdc[3222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:06 dist-test-slave-vxj2 krb5kdc[3222](info): set up 2 sockets
Aug 10 06:49:06 dist-test-slave-vxj2 krb5kdc[3222](info): commencing operation
krb5kdc: starting...
W20260810 06:49:07.523682 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:07 dist-test-slave-vxj2 krb5kdc[3222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:49:07.657101 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:35631
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:35631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:07.742971 3239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:07.743207 3239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:07.743238 3239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:07.744344 3239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:07.744378 3239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:07.744392 3239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:07.744405 3239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:07.744417 3239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:07.745898 3239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:35631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:35631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:07.746110 3239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:07.746305 3239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:07.752063 3245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:07.752004 3244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:07.751996 3239 server_base.cc:1034] running on GCE node
W20260810 06:49:07.751991 3247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:07.752578 3239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:07.752954 3239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:07.754145 3239 hybrid_clock.cc:648] HybridClock initialized: now 1786344547754120 us; error 43 us; skew 500 ppm
Aug 10 06:49:07 dist-test-slave-vxj2 krb5kdc[3222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344547, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:07.757036 3239 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:49:07.757575 3239 webserver.cc:460] Webserver started at http://127.15.232.254:39775/ using document root <none> and password file <none>
I20260810 06:49:07.757833 3239 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:07.757879 3239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:07.758054 3239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:07.759137 3239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "7b7945133b7041c0ae589ddc62d13bbe"
format_stamp: "Formatted at 2026-08-10 06:49:07 on dist-test-slave-vxj2"
server_key: "310d2d51b088214c1d87998d13334999"
server_key_iv: "f67f524ea936fe31708cb05ad9e95349"
server_key_version: "encryptionkey@0"
I20260810 06:49:07.759550 3239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "7b7945133b7041c0ae589ddc62d13bbe"
format_stamp: "Formatted at 2026-08-10 06:49:07 on dist-test-slave-vxj2"
server_key: "310d2d51b088214c1d87998d13334999"
server_key_iv: "f67f524ea936fe31708cb05ad9e95349"
server_key_version: "encryptionkey@0"
I20260810 06:49:07.761128 3239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:49:07.762184 3256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:07.762429 3239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:07.762554 3239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "7b7945133b7041c0ae589ddc62d13bbe"
format_stamp: "Formatted at 2026-08-10 06:49:07 on dist-test-slave-vxj2"
server_key: "310d2d51b088214c1d87998d13334999"
server_key_iv: "f67f524ea936fe31708cb05ad9e95349"
server_key_version: "encryptionkey@0"
I20260810 06:49:07.762665 3239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:07.779091 3239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:07.781585 3239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:07.781839 3239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:07.786187 3239 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:35631
I20260810 06:49:07.786238 3315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:35631 every 8 connection(s)
I20260810 06:49:07.786653 3239 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:07.787559 3316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:07.790118 3316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe: Bootstrap starting.
I20260810 06:49:07.790743 3316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:07.791054 3316 log.cc:824] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:07.791805 3316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe: No bootstrap required, opened a new log
I20260810 06:49:07.792997 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 3239
I20260810 06:49:07.793145 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:07.793375 16291 external_mini_cluster.cc:1428] Setting key 1b27077b9aa20b6637adb3a7391963b3
I20260810 06:49:07.793668 3316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7945133b7041c0ae589ddc62d13bbe" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35631 } }
I20260810 06:49:07.793893 3316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:07.793982 3316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b7945133b7041c0ae589ddc62d13bbe, State: Initialized, Role: FOLLOWER
I20260810 06:49:07.794137 3316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7945133b7041c0ae589ddc62d13bbe" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35631 } }
I20260810 06:49:07.794242 3316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:07.794292 3316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:07.794345 3316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:07.794921 3316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7945133b7041c0ae589ddc62d13bbe" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35631 } }
I20260810 06:49:07.795045 3316 leader_election.cc:302] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b7945133b7041c0ae589ddc62d13bbe; no voters:
I20260810 06:49:07.795252 3316 leader_election.cc:288] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:07.795274 3321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:07.795389 3321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [term 1 LEADER]: Becoming Leader. State: Replica: 7b7945133b7041c0ae589ddc62d13bbe, State: Running, Role: LEADER
I20260810 06:49:07.795480 3321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7945133b7041c0ae589ddc62d13bbe" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35631 } }
I20260810 06:49:07.795532 3316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:07.795814 3321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b7945133b7041c0ae589ddc62d13bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7945133b7041c0ae589ddc62d13bbe" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35631 } } }
I20260810 06:49:07.795979 3321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:07.796176 3321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b7945133b7041c0ae589ddc62d13bbe. Latest consensus state: current_term: 1 leader_uuid: "7b7945133b7041c0ae589ddc62d13bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7945133b7041c0ae589ddc62d13bbe" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35631 } } }
I20260810 06:49:07.796275 3321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:07.796686 3331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:07.797246 3331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:07.799013 3331 catalog_manager.cc:1347] Generated new cluster ID: 8ca170a03011414a98110ce7ad96a372
I20260810 06:49:07.799115 3331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:49:07 dist-test-slave-vxj2 krb5kdc[3222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:07.804955 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:07.793969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44808 (local address 127.15.232.254:35631)
0810 06:49:07.794191 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:07.794199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:07.794225 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:49:07.794375 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:07.794382 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:07.794448 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:07.794589 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:07.795436 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:07.796952 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:07.797898 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:07.798535 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:07.798666 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:07.801363 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:07.801369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:07.801369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:07.801387 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:07.803275 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:07.803691 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:07.803702 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:07.803703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:07.803810 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:07.804033 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:07.804034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:07.804035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:07.804306 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:07.804505 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:07.804576 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 06:49:07.804732 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":93,"threads_started":1}
I20260810 06:49:07.818789 3331 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:07.819430 3331 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:07.822487 3331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe: Generated new TSK 0
I20260810 06:49:07.822701 3331 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:49:07.856853 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35631
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:07.958230 3345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:07.958457 3345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:07.958482 3345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:07.959594 3345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:07.959628 3345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:07.959654 3345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:49:07.961194 3345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:07.961419 3345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:07.961606 3345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:07.966795 3354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:07.966933 3351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:07.966933 3345 server_base.cc:1034] running on GCE node
W20260810 06:49:07.966871 3352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:07.967360 3345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:07.967696 3345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:07.968894 3345 hybrid_clock.cc:648] HybridClock initialized: now 1786344547968883 us; error 43 us; skew 500 ppm
Aug 10 06:49:07 dist-test-slave-vxj2 krb5kdc[3222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344547, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:07.971935 3345 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:49:07.972415 3345 webserver.cc:460] Webserver started at http://127.15.232.193:35463/ using document root <none> and password file <none>
I20260810 06:49:07.972673 3345 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:07.972745 3345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:07.972965 3345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:07.974201 3345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "e1185a63b2e243858451af28d5e6e4b7"
format_stamp: "Formatted at 2026-08-10 06:49:07 on dist-test-slave-vxj2"
server_key: "8977ea6bf254657d7134159386fd9b7f"
server_key_iv: "9ccb22371746684345778ba28cd9f5fa"
server_key_version: "encryptionkey@0"
I20260810 06:49:07.974591 3345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "e1185a63b2e243858451af28d5e6e4b7"
format_stamp: "Formatted at 2026-08-10 06:49:07 on dist-test-slave-vxj2"
server_key: "8977ea6bf254657d7134159386fd9b7f"
server_key_iv: "9ccb22371746684345778ba28cd9f5fa"
server_key_version: "encryptionkey@0"
I20260810 06:49:07.976009 3345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:07.976897 3364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:07.977125 3345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:07.977319 3345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "e1185a63b2e243858451af28d5e6e4b7"
format_stamp: "Formatted at 2026-08-10 06:49:07 on dist-test-slave-vxj2"
server_key: "8977ea6bf254657d7134159386fd9b7f"
server_key_iv: "9ccb22371746684345778ba28cd9f5fa"
server_key_version: "encryptionkey@0"
I20260810 06:49:07.977429 3345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:07.998376 3345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:08.001046 3345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:08.001296 3345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:08.001581 3345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:08.001929 3345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:08.001988 3345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:08.002035 3345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:08.002060 3345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:08.008138 3345 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:37161
I20260810 06:49:08.008244 3499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:37161 every 8 connection(s)
I20260810 06:49:08.008620 3345 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:49:08.013680 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 3345
I20260810 06:49:08.013898 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:49:08.014165 16291 external_mini_cluster.cc:1428] Setting key a35dc041d87e4f575b1e3fb9acd7b155
Aug 10 06:49:08 dist-test-slave-vxj2 krb5kdc[3222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344547, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:08.019101 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.009495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:56485 (local address 127.15.232.254:35631)
0810 06:49:08.009571 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:08.009573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:08.009801 (+ 228us) server_negotiation.cc:407] Connection header received
0810 06:49:08.010291 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:08.010293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:08.010307 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:08.010377 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:08.011432 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.011970 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.013020 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.013235 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.013290 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.015856 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:08.015857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:08.015858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:08.015866 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:08.017466 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:08.018001 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:08.018002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:08.018002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:08.018058 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:08.018355 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:08.018356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:08.018356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:08.018471 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:08.018530 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:08.018935 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 06:49:08.019030 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:49:08.019363 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.009478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35631 (local address 127.15.232.193:56485)
0810 06:49:08.009654 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:08.009701 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:08.010189 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:08.010443 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:08.010446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:08.011109 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:08.011328 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.011333 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.012069 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.012070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.012925 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.012928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.013309 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.013311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.013419 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.014132 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:08.014144 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:08.015736 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:08.017551 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:08.017559 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:08.017561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:08.017922 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:08.018126 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:08.018127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:08.018128 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:08.018282 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:08.018547 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:08.018555 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:08.018743 (+ 188us) client_negotiation.cc:769] Sending connection context
0810 06:49:08.018887 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 06:49:08.019093 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
I20260810 06:49:08.020023 3500 heartbeater.cc:347] Connected to a master server at 127.15.232.254:35631
I20260810 06:49:08.020133 3500 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:08.020351 3500 heartbeater.cc:511] Master 127.15.232.254:35631 requested a full tablet report, sending...
I20260810 06:49:08.020917 3275 ts_manager.cc:194] Registered new tserver with Master: e1185a63b2e243858451af28d5e6e4b7 (127.15.232.193:37161)
I20260810 06:49:08.021625 3275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:56485
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:49:08.045744 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35631
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:08.131217 3506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:08.131459 3506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:08.131487 3506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:08.132632 3506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:08.132669 3506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:08.132697 3506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:49:08.134248 3506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:08.134482 3506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:08.134675 3506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:08.140157 3514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:08.140178 3511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:08.140439 3506 server_base.cc:1034] running on GCE node
W20260810 06:49:08.140182 3516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:08.140663 3506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:08.140942 3506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:08.142122 3506 hybrid_clock.cc:648] HybridClock initialized: now 1786344548142099 us; error 55 us; skew 500 ppm
Aug 10 06:49:08 dist-test-slave-vxj2 krb5kdc[3222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344548, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:08.144904 3506 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:49:08.145453 3506 webserver.cc:460] Webserver started at http://127.15.232.194:32825/ using document root <none> and password file <none>
I20260810 06:49:08.145709 3506 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:08.145785 3506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:08.145948 3506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:08.147053 3506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "c2d2242bcd464a3bb0c6732e45e2c025"
format_stamp: "Formatted at 2026-08-10 06:49:08 on dist-test-slave-vxj2"
server_key: "42128afe39aef00f0949d2e32ac613c5"
server_key_iv: "7e6233ae55b26b048de053daf4eb7f9b"
server_key_version: "encryptionkey@0"
I20260810 06:49:08.147430 3506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "c2d2242bcd464a3bb0c6732e45e2c025"
format_stamp: "Formatted at 2026-08-10 06:49:08 on dist-test-slave-vxj2"
server_key: "42128afe39aef00f0949d2e32ac613c5"
server_key_iv: "7e6233ae55b26b048de053daf4eb7f9b"
server_key_version: "encryptionkey@0"
I20260810 06:49:08.148905 3506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:08.149909 3523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:08.150211 3506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:08.150379 3506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "c2d2242bcd464a3bb0c6732e45e2c025"
format_stamp: "Formatted at 2026-08-10 06:49:08 on dist-test-slave-vxj2"
server_key: "42128afe39aef00f0949d2e32ac613c5"
server_key_iv: "7e6233ae55b26b048de053daf4eb7f9b"
server_key_version: "encryptionkey@0"
I20260810 06:49:08.150472 3506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:08.178858 3506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:08.181492 3506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:08.181699 3506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:08.182044 3506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:08.182410 3506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:08.182472 3506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:08.182523 3506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:08.182554 3506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:08.189567 3506 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:41187
I20260810 06:49:08.189601 3659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:41187 every 8 connection(s)
I20260810 06:49:08.190007 3506 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:49:08.191684 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 3506
I20260810 06:49:08.191880 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:49:08.192124 16291 external_mini_cluster.cc:1428] Setting key 6838a0d41384da252363f8c900ec39ef
Aug 10 06:49:08 dist-test-slave-vxj2 krb5kdc[3222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344548, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
I20260810 06:49:08.202142 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.190900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:46473 (local address 127.15.232.254:35631)
0810 06:49:08.190959 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:08.190961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:08.191174 (+ 213us) server_negotiation.cc:407] Connection header received
0810 06:49:08.191733 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:08.191734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:08.191748 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:08.191820 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:08.193253 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.193812 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.195306 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.195517 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.195597 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.198392 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:08.198393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:08.198394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:08.198403 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:08.200295 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:08.200826 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:08.200827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:08.200828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:08.200896 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:08.201260 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:08.201262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:08.201262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:08.201424 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:08.201513 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:08.201885 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 06:49:08.202007 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:49:08.202267 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.190818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35631 (local address 127.15.232.194:46473)
0810 06:49:08.191025 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:08.191074 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:08.191605 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:08.191941 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:08.191947 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:08.192822 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:08.193106 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.193114 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.193921 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.193922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.195197 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.195200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.195603 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.195605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.195744 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.196477 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:08.196487 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:08.198281 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:08.200398 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:08.200406 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:08.200408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:08.200672 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:08.200991 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:08.200997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:08.200998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:08.201112 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:08.201522 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:08.201529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:08.201669 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 06:49:08.201812 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 06:49:08.201957 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":127,"threads_started":1}
I20260810 06:49:08.202924 3660 heartbeater.cc:347] Connected to a master server at 127.15.232.254:35631
I20260810 06:49:08.203020 3660 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:08.203338 3660 heartbeater.cc:511] Master 127.15.232.254:35631 requested a full tablet report, sending...
I20260810 06:49:08.203833 3275 ts_manager.cc:194] Registered new tserver with Master: c2d2242bcd464a3bb0c6732e45e2c025 (127.15.232.194:41187)
I20260810 06:49:08.204329 3275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:46473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:49:08.224452 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35631
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:08.315333 3665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:08.315562 3665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:08.315588 3665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:08.316723 3665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:08.316762 3665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:08.316793 3665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:49:08.318343 3665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:35631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:08.318558 3665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:08.318753 3665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:08.324451 3672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:08.324505 3665 server_base.cc:1034] running on GCE node
W20260810 06:49:08.324472 3671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:08.324476 3674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:08.324961 3665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:08.325289 3665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:08.326500 3665 hybrid_clock.cc:648] HybridClock initialized: now 1786344548326486 us; error 31 us; skew 500 ppm
Aug 10 06:49:08 dist-test-slave-vxj2 krb5kdc[3222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344548, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:08.329339 3665 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:49:08.329798 3665 webserver.cc:460] Webserver started at http://127.15.232.195:38189/ using document root <none> and password file <none>
I20260810 06:49:08.329993 3665 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:08.330072 3665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:08.330245 3665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:08.331346 3665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "84572f3796da4fb295a773eac07aa0d0"
format_stamp: "Formatted at 2026-08-10 06:49:08 on dist-test-slave-vxj2"
server_key: "4446d6a404a362ef67db866d20915e25"
server_key_iv: "1f51a1a6193d609f40665c18fbb527ec"
server_key_version: "encryptionkey@0"
I20260810 06:49:08.331730 3665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "84572f3796da4fb295a773eac07aa0d0"
format_stamp: "Formatted at 2026-08-10 06:49:08 on dist-test-slave-vxj2"
server_key: "4446d6a404a362ef67db866d20915e25"
server_key_iv: "1f51a1a6193d609f40665c18fbb527ec"
server_key_version: "encryptionkey@0"
I20260810 06:49:08.333139 3665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:49:08.334087 3684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:08.334368 3665 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:49:08.334483 3665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "84572f3796da4fb295a773eac07aa0d0"
format_stamp: "Formatted at 2026-08-10 06:49:08 on dist-test-slave-vxj2"
server_key: "4446d6a404a362ef67db866d20915e25"
server_key_iv: "1f51a1a6193d609f40665c18fbb527ec"
server_key_version: "encryptionkey@0"
I20260810 06:49:08.334561 3665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:08.380631 3665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:08.383288 3665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:08.383474 3665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:08.383769 3665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:08.384176 3665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:08.384235 3665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:08.384270 3665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:08.384298 3665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:08.391101 3665 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:41789
I20260810 06:49:08.391173 3816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:41789 every 8 connection(s)
I20260810 06:49:08.391562 3665 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:49:08 dist-test-slave-vxj2 krb5kdc[3222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344548, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:08.401306 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 3665
I20260810 06:49:08.401424 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:49:08.401608 16291 external_mini_cluster.cc:1428] Setting key 6e6cfc8e2e8948c54df1ac470abb740f
I20260810 06:49:08.401960 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.392405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:36589 (local address 127.15.232.254:35631)
0810 06:49:08.392469 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:08.392471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:08.392656 (+ 185us) server_negotiation.cc:407] Connection header received
0810 06:49:08.393327 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:08.393328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:08.393343 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:08.393402 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:08.394433 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.394952 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.396081 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.396315 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.396422 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.398888 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:08.398889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:08.398890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:08.398897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:08.400175 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:08.400636 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:08.400638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:08.400638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:08.400700 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:08.401047 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:08.401048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:08.401049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:08.401256 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:08.401318 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:08.401718 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 06:49:08.401851 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:49:08.402206 3820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.392329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:35631 (local address 127.15.232.195:36589)
0810 06:49:08.392514 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:08.392560 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:08.393199 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:08.393489 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:08.393493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:08.394117 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:08.394326 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.394330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.395062 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.395064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.395984 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.395987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.396472 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.396473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.396648 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.397288 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:08.397297 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:08.398787 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:08.400270 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:08.400278 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:08.400280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:08.400534 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:08.400796 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:08.400797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:08.400798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:08.400946 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:08.401328 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:08.401337 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:08.401539 (+ 202us) client_negotiation.cc:769] Sending connection context
0810 06:49:08.401675 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 06:49:08.401894 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260810 06:49:08.402678 3817 heartbeater.cc:347] Connected to a master server at 127.15.232.254:35631
I20260810 06:49:08.402803 3817 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:08.402973 3817 heartbeater.cc:511] Master 127.15.232.254:35631 requested a full tablet report, sending...
I20260810 06:49:08.403381 3275 ts_manager.cc:194] Registered new tserver with Master: 84572f3796da4fb295a773eac07aa0d0 (127.15.232.195:41789)
I20260810 06:49:08.403786 3275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:36589
I20260810 06:49:08.413959 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:49:08.421691 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.415365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44820 (local address 127.15.232.254:35631)
0810 06:49:08.415453 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:08.415456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:08.415548 (+ 92us) server_negotiation.cc:407] Connection header received
0810 06:49:08.415588 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:08.415588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:08.415598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:08.415655 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:08.416201 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.416697 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.417514 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.417703 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.417761 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.419037 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:08.419039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:08.419039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:08.419046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:08.420507 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:08.420885 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:08.420886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:08.420886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:08.420942 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:08.421120 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:08.421121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:08.421121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:08.421347 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:08.421429 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:08.421526 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 06:49:08.421591 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:49:08.423530 3275 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:49:08.425287 3275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:08.433036 3830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.428372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:41187 (local address 127.0.0.1:35744)
0810 06:49:08.428687 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:08.428764 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:08.428893 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:08.429224 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:08.429225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:08.429227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:08.429268 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.429269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.430320 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.430321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.431642 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.431645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.432614 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.432615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.432693 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.432705 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 06:49:08.432875 (+ 170us) client_negotiation.cc:240] Negotiation successful
0810 06:49:08.432949 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":66,"threads_started":1}
I20260810 06:49:08.433050 3833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.428469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35744 (local address 127.15.232.194:41187)
0810 06:49:08.428682 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:08.428688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:08.428846 (+ 158us) server_negotiation.cc:407] Connection header received
0810 06:49:08.429004 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:08.429006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:08.429020 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:08.429101 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:08.429357 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.430198 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.431745 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.432522 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.432604 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.432667 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:08.432899 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 06:49:08.432948 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":102,"threads_started":1}
I20260810 06:49:08.433736 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.428423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:37161 (local address 127.0.0.1:33702)
0810 06:49:08.429232 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:08.429241 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:08.429327 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:08.429552 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:08.429553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:08.429555 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:08.429594 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.429595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.430944 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.430944 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.431725 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.431729 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.433503 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.433503 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.433569 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.433573 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:49:08.433624 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 06:49:08.433665 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":75,"threads_started":1}
I20260810 06:49:08.433800 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.428652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33702 (local address 127.15.232.193:37161)
0810 06:49:08.429142 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:08.429146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:08.429395 (+ 249us) server_negotiation.cc:407] Connection header received
0810 06:49:08.429439 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:08.429442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:08.429456 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:08.429531 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:08.429665 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.430731 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.431831 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.432522 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.432605 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.432667 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:08.433658 (+ 991us) server_negotiation.cc:299] Negotiation successful
0810 06:49:08.433707 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":86,"threads_started":1}
I20260810 06:49:08.433915 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.428636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32848 (local address 127.15.232.195:41789)
0810 06:49:08.428920 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:08.428925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:08.428935 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:49:08.428980 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:08.428982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:08.429001 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:08.429092 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:08.429328 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.430501 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.432710 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.433412 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.433492 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.433550 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:08.433773 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 06:49:08.433824 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":99,"threads_started":1}
I20260810 06:49:08.433959 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.428387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:41789 (local address 127.0.0.1:32848)
0810 06:49:08.428825 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:08.428835 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:08.428893 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:08.429106 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:08.429109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:08.429111 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:08.429231 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.429234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.430613 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.430614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.431647 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.431649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.433537 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.433538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.433609 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.433614 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:49:08.433665 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 06:49:08.433707 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":95,"threads_started":1}
I20260810 06:49:08.434006 3577 tablet_service.cc:1467] Processing CreateTablet for tablet 52b816c682ae45698bfbf241f790581d (DEFAULT_TABLE table=test-table [id=faa5c951b28a498b95128d732b5d5471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:08.434412 3577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52b816c682ae45698bfbf241f790581d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:08.434463 3419 tablet_service.cc:1467] Processing CreateTablet for tablet 52b816c682ae45698bfbf241f790581d (DEFAULT_TABLE table=test-table [id=faa5c951b28a498b95128d732b5d5471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:08.434794 3419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52b816c682ae45698bfbf241f790581d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:08.437939 3743 tablet_service.cc:1467] Processing CreateTablet for tablet 52b816c682ae45698bfbf241f790581d (DEFAULT_TABLE table=test-table [id=faa5c951b28a498b95128d732b5d5471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:08.438341 3836 tablet_bootstrap.cc:492] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025: Bootstrap starting.
I20260810 06:49:08.438386 3743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52b816c682ae45698bfbf241f790581d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:08.439147 3836 tablet_bootstrap.cc:654] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:08.439785 3836 log.cc:824] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:08.440162 3837 tablet_bootstrap.cc:492] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7: Bootstrap starting.
I20260810 06:49:08.440718 3836 tablet_bootstrap.cc:492] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025: No bootstrap required, opened a new log
I20260810 06:49:08.440814 3836 ts_tablet_manager.cc:1397] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:49:08.440881 3837 tablet_bootstrap.cc:654] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:08.441516 3837 log.cc:824] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:08.442378 3837 tablet_bootstrap.cc:492] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7: No bootstrap required, opened a new log
I20260810 06:49:08.442456 3837 ts_tablet_manager.cc:1397] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260810 06:49:08.442679 3840 tablet_bootstrap.cc:492] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0: Bootstrap starting.
I20260810 06:49:08.442751 3836 raft_consensus.cc:348] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 41187 } } peers { permanent_uuid: "84572f3796da4fb295a773eac07aa0d0" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 41789 } } peers { permanent_uuid: "e1185a63b2e243858451af28d5e6e4b7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37161 } }
I20260810 06:49:08.442996 3836 raft_consensus.cc:374] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:08.443073 3836 raft_consensus.cc:729] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d2242bcd464a3bb0c6732e45e2c025, State: Initialized, Role: FOLLOWER
I20260810 06:49:08.443252 3836 consensus_queue.cc:260] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 41187 } } peers { permanent_uuid: "84572f3796da4fb295a773eac07aa0d0" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 41789 } } peers { permanent_uuid: "e1185a63b2e243858451af28d5e6e4b7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37161 } }
I20260810 06:49:08.443466 3840 tablet_bootstrap.cc:654] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:08.443619 3660 heartbeater.cc:503] Master 127.15.232.254:35631 was elected leader, sending a full tablet report...
I20260810 06:49:08.443609 3836 ts_tablet_manager.cc:1428] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:49:08.444165 3840 log.cc:824] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:08.444681 3837 raft_consensus.cc:348] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 41187 } } peers { permanent_uuid: "84572f3796da4fb295a773eac07aa0d0" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 41789 } } peers { permanent_uuid: "e1185a63b2e243858451af28d5e6e4b7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37161 } }
I20260810 06:49:08.444949 3837 raft_consensus.cc:374] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:08.445011 3837 raft_consensus.cc:729] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1185a63b2e243858451af28d5e6e4b7, State: Initialized, Role: FOLLOWER
I20260810 06:49:08.445053 3840 tablet_bootstrap.cc:492] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0: No bootstrap required, opened a new log
I20260810 06:49:08.445190 3840 ts_tablet_manager.cc:1397] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:49:08.445140 3837 consensus_queue.cc:260] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 41187 } } peers { permanent_uuid: "84572f3796da4fb295a773eac07aa0d0" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 41789 } } peers { permanent_uuid: "e1185a63b2e243858451af28d5e6e4b7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37161 } }
I20260810 06:49:08.445505 3837 ts_tablet_manager.cc:1428] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:49:08.445531 3500 heartbeater.cc:503] Master 127.15.232.254:35631 was elected leader, sending a full tablet report...
I20260810 06:49:08.447088 3840 raft_consensus.cc:348] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 41187 } } peers { permanent_uuid: "84572f3796da4fb295a773eac07aa0d0" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 41789 } } peers { permanent_uuid: "e1185a63b2e243858451af28d5e6e4b7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37161 } }
I20260810 06:49:08.447218 3840 raft_consensus.cc:374] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:08.447240 3840 raft_consensus.cc:729] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84572f3796da4fb295a773eac07aa0d0, State: Initialized, Role: FOLLOWER
I20260810 06:49:08.447378 3840 consensus_queue.cc:260] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 41187 } } peers { permanent_uuid: "84572f3796da4fb295a773eac07aa0d0" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 41789 } } peers { permanent_uuid: "e1185a63b2e243858451af28d5e6e4b7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37161 } }
I20260810 06:49:08.447670 3840 ts_tablet_manager.cc:1428] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:49:08.447745 3817 heartbeater.cc:503] Master 127.15.232.254:35631 was elected leader, sending a full tablet report...
W20260810 06:49:08.509404 3501 tablet.cc:2369] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:49:08.642202 3819 tablet.cc:2369] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:49:08.690827 3661 tablet.cc:2369] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:49:08.749439 3842 raft_consensus.cc:482] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:08.749578 3842 raft_consensus.cc:504] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 41187 } } peers { permanent_uuid: "84572f3796da4fb295a773eac07aa0d0" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 41789 } } peers { permanent_uuid: "e1185a63b2e243858451af28d5e6e4b7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37161 } }
I20260810 06:49:08.750065 3842 leader_election.cc:288] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84572f3796da4fb295a773eac07aa0d0 (127.15.232.195:41789), e1185a63b2e243858451af28d5e6e4b7 (127.15.232.193:37161)
I20260810 06:49:08.753294 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.750400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:33653 (local address 127.15.232.193:37161)
0810 06:49:08.750457 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:08.750460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:08.750712 (+ 252us) server_negotiation.cc:407] Connection header received
0810 06:49:08.750795 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:08.750796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:08.750807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:08.750852 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:08.751033 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.751517 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.752528 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.752885 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.752995 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.753004 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:08.753183 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 06:49:08.753219 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 06:49:08.753309 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.750514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:39181 (local address 127.15.232.195:41789)
0810 06:49:08.750576 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:08.750578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:08.750591 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:49:08.750631 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:08.750632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:08.750644 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:08.750714 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:08.751065 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.751554 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.752536 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.752921 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.753007 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.753015 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:08.753185 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 06:49:08.753226 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:49:08.753340 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.750311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:41789 (local address 127.15.232.194:39181)
0810 06:49:08.750520 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:08.750530 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:08.750582 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:08.750903 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:08.750903 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:08.750905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:08.750980 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.750981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.751646 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.751646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.752440 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.752442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.753008 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.753009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.753087 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.753090 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:49:08.753158 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:49:08.753237 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":90,"threads_started":1}
I20260810 06:49:08.753343 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.750313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:37161 (local address 127.15.232.194:33653)
0810 06:49:08.750613 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:08.750624 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:08.750705 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:08.750877 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:08.750878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:08.750880 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:08.750946 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.750948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.751603 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.751604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.752431 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:08.752433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.753005 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.753006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:08.753085 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.753088 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:49:08.753158 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:49:08.753216 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":38,"thread_start_us":78,"threads_started":1}
I20260810 06:49:08.753758 3440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b816c682ae45698bfbf241f790581d" candidate_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1185a63b2e243858451af28d5e6e4b7" is_pre_election: true
I20260810 06:49:08.753789 3766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b816c682ae45698bfbf241f790581d" candidate_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84572f3796da4fb295a773eac07aa0d0" is_pre_election: true
I20260810 06:49:08.753928 3766 raft_consensus.cc:2454] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2d2242bcd464a3bb0c6732e45e2c025 in term 0.
I20260810 06:49:08.753928 3440 raft_consensus.cc:2454] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2d2242bcd464a3bb0c6732e45e2c025 in term 0.
I20260810 06:49:08.754184 3527 leader_election.cc:302] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2d2242bcd464a3bb0c6732e45e2c025, e1185a63b2e243858451af28d5e6e4b7; no voters:
I20260810 06:49:08.754367 3842 raft_consensus.cc:2781] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:08.754475 3842 raft_consensus.cc:482] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:08.754509 3842 raft_consensus.cc:3037] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:08.755357 3842 raft_consensus.cc:504] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 41187 } } peers { permanent_uuid: "84572f3796da4fb295a773eac07aa0d0" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 41789 } } peers { permanent_uuid: "e1185a63b2e243858451af28d5e6e4b7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37161 } }
I20260810 06:49:08.755519 3842 leader_election.cc:288] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [CANDIDATE]: Term 1 election: Requested vote from peers 84572f3796da4fb295a773eac07aa0d0 (127.15.232.195:41789), e1185a63b2e243858451af28d5e6e4b7 (127.15.232.193:37161)
I20260810 06:49:08.755707 3440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b816c682ae45698bfbf241f790581d" candidate_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1185a63b2e243858451af28d5e6e4b7"
I20260810 06:49:08.755728 3766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b816c682ae45698bfbf241f790581d" candidate_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84572f3796da4fb295a773eac07aa0d0"
I20260810 06:49:08.755790 3440 raft_consensus.cc:3037] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:08.755836 3766 raft_consensus.cc:3037] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:08.756527 3440 raft_consensus.cc:2454] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2d2242bcd464a3bb0c6732e45e2c025 in term 1.
I20260810 06:49:08.756709 3766 raft_consensus.cc:2454] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2d2242bcd464a3bb0c6732e45e2c025 in term 1.
I20260810 06:49:08.756723 3527 leader_election.cc:302] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2d2242bcd464a3bb0c6732e45e2c025, e1185a63b2e243858451af28d5e6e4b7; no voters:
I20260810 06:49:08.756807 3842 raft_consensus.cc:2781] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:08.756958 3842 raft_consensus.cc:686] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 1 LEADER]: Becoming Leader. State: Replica: c2d2242bcd464a3bb0c6732e45e2c025, State: Running, Role: LEADER
I20260810 06:49:08.757079 3842 consensus_queue.cc:237] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 41187 } } peers { permanent_uuid: "84572f3796da4fb295a773eac07aa0d0" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 41789 } } peers { permanent_uuid: "e1185a63b2e243858451af28d5e6e4b7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37161 } }
I20260810 06:49:08.758026 3274 catalog_manager.cc:5495] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 reported cstate change: term changed from 0 to 1, leader changed from <none> to c2d2242bcd464a3bb0c6732e45e2c025 (127.15.232.194). New cstate: current_term: 1 leader_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d2242bcd464a3bb0c6732e45e2c025" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 41187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84572f3796da4fb295a773eac07aa0d0" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 41789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1185a63b2e243858451af28d5e6e4b7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37161 } health_report { overall_health: UNKNOWN } } }
I20260810 06:49:08.787564 3833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.784860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35746 (local address 127.15.232.194:41187)
0810 06:49:08.784925 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:08.784927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:08.784937 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:49:08.785008 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:08.785009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:08.785030 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:08.785101 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:08.785308 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.785790 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.786721 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:08.786961 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:08.787044 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:08.787211 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:08.787320 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:08.787419 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:08.787438 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 06:49:08.787461 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:49:08.789566 3766 raft_consensus.cc:1261] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0 [term 1 FOLLOWER]: Refusing update from remote peer c2d2242bcd464a3bb0c6732e45e2c025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:08.789565 3440 raft_consensus.cc:1261] T 52b816c682ae45698bfbf241f790581d P e1185a63b2e243858451af28d5e6e4b7 [term 1 FOLLOWER]: Refusing update from remote peer c2d2242bcd464a3bb0c6732e45e2c025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:08.789956 3848 consensus_queue.cc:1035] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84572f3796da4fb295a773eac07aa0d0" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 41789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:08.790132 3842 consensus_queue.cc:1035] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1185a63b2e243858451af28d5e6e4b7" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 37161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:08.792519 3852 mvcc.cc:205] Tried to move back new op lower bound from 7316867271838945280 to 7316867271709847552. Current Snapshot: MvccSnapshot[applied={T|T < 7316867271838945280}]
I20260810 06:49:08.792883 3853 mvcc.cc:205] Tried to move back new op lower bound from 7316867271838945280 to 7316867271709847552. Current Snapshot: MvccSnapshot[applied={T|T < 7316867271838945280}]
I20260810 06:49:08.795619 3274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:08.795706 3274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:08.796559 3274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b7945133b7041c0ae589ddc62d13bbe: Sending DeleteTablet for 3 replicas of tablet 52b816c682ae45698bfbf241f790581d
I20260810 06:49:08.796960 3743 tablet_service.cc:1514] Processing DeleteTablet for tablet 52b816c682ae45698bfbf241f790581d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:08 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:32848
I20260810 06:49:08.796981 3577 tablet_service.cc:1514] Processing DeleteTablet for tablet 52b816c682ae45698bfbf241f790581d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:08 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:35744
I20260810 06:49:08.797219 3419 tablet_service.cc:1514] Processing DeleteTablet for tablet 52b816c682ae45698bfbf241f790581d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:08 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:33702
I20260810 06:49:08.797400 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 3345
I20260810 06:49:08.799906 3865 tablet_replica.cc:330] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0: stopping tablet replica
I20260810 06:49:08.800060 3865 raft_consensus.cc:2229] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:08.800237 3865 raft_consensus.cc:2258] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:08.801282 3865 ts_tablet_manager.cc:1905] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:08.803604 3865 ts_tablet_manager.cc:1918] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:08.803977 3865 log.cc:1196] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/52b816c682ae45698bfbf241f790581d
I20260810 06:49:08.804394 3865 ts_tablet_manager.cc:1939] T 52b816c682ae45698bfbf241f790581d P 84572f3796da4fb295a773eac07aa0d0: Deleting consensus metadata
I20260810 06:49:08.804934 3258 catalog_manager.cc:4841] TS 84572f3796da4fb295a773eac07aa0d0 (127.15.232.195:41789): tablet 52b816c682ae45698bfbf241f790581d (table test-table [id=faa5c951b28a498b95128d732b5d5471]) successfully deleted
I20260810 06:49:08.805333 3866 tablet_replica.cc:330] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025: stopping tablet replica
I20260810 06:49:08.805486 3866 raft_consensus.cc:2229] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:49:08.805616 3866 raft_consensus.cc:2258] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:08.806367 3866 ts_tablet_manager.cc:1905] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 06:49:08.807818 3259 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:37161 (error 108)
I20260810 06:49:08.808112 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 3506
I20260810 06:49:08.808190 3866 ts_tablet_manager.cc:1918] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:08.808267 3866 log.cc:1196] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/52b816c682ae45698bfbf241f790581d
I20260810 06:49:08.808558 3866 ts_tablet_manager.cc:1939] T 52b816c682ae45698bfbf241f790581d P c2d2242bcd464a3bb0c6732e45e2c025: Deleting consensus metadata
I20260810 06:49:08.809830 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.809661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:37161 (local address 127.0.0.1:33712)
0810 06:49:08.809741 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:08.809786 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 06:49:08.809995 3259 catalog_manager.cc:4584] TS e1185a63b2e243858451af28d5e6e4b7 (127.15.232.193:37161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52b816c682ae45698bfbf241f790581d: Network error: Client connection negotiation failed: client connection to 127.15.232.193:37161: connect: Connection refused (error 111)
I20260810 06:49:08.818069 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 3665
I20260810 06:49:08.818249 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.818064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:41187 (local address 127.0.0.1:35748)
0810 06:49:08.818147 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:08.818190 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.194:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 06:49:08.827661 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 3239
2026-08-10T06:49:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2464 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T06:49:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:08Z Disabled control of system clock
I20260810 06:49:08.861019 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:41039
--webserver_interface=127.15.232.254
--webserver_port=0
--builtin_ntp_servers=127.15.232.212:39937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:41039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:49:08.946388 3875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:08.946630 3875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:08.946651 3875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:08.947824 3875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:08.947870 3875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:08.947883 3875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:08.947896 3875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:08.947908 3875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:08.949398 3875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:39937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:41039
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:41039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:08.949679 3875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:08.949995 3875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:49:08.955049 3875 server_base.cc:1034] running on GCE node
W20260810 06:49:08.955049 3881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:08.955227 3880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:08.955111 3884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:08.955611 3875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:08.955890 3875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:08.957033 3875 hybrid_clock.cc:648] HybridClock initialized: now 1786344548957020 us; error 38 us; skew 500 ppm
I20260810 06:49:08.958374 3875 webserver.cc:460] Webserver started at http://127.15.232.254:42079/ using document root <none> and password file <none>
I20260810 06:49:08.958654 3875 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:08.958706 3875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:08.958881 3875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:08.960038 3875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "017b70c94586408a9c593aa816f31c31"
format_stamp: "Formatted at 2026-08-10 06:49:08 on dist-test-slave-vxj2"
server_key: "036d428641a0f1e09108e66dd44cb7c2"
server_key_iv: "6d1f84311a85b430232ea76fbb0ee2dd"
server_key_version: "encryptionkey@0"
I20260810 06:49:08.960428 3875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "017b70c94586408a9c593aa816f31c31"
format_stamp: "Formatted at 2026-08-10 06:49:08 on dist-test-slave-vxj2"
server_key: "036d428641a0f1e09108e66dd44cb7c2"
server_key_iv: "6d1f84311a85b430232ea76fbb0ee2dd"
server_key_version: "encryptionkey@0"
I20260810 06:49:08.961885 3875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 06:49:08.962693 3890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:08.962939 3875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:08.963042 3875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "017b70c94586408a9c593aa816f31c31"
format_stamp: "Formatted at 2026-08-10 06:49:08 on dist-test-slave-vxj2"
server_key: "036d428641a0f1e09108e66dd44cb7c2"
server_key_iv: "6d1f84311a85b430232ea76fbb0ee2dd"
server_key_version: "encryptionkey@0"
I20260810 06:49:08.963101 3875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:08.982353 3875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:08.982702 3875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:08.982827 3875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:08.987107 3875 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:41039
I20260810 06:49:08.987141 3947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:41039 every 8 connection(s)
I20260810 06:49:08.987560 3875 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:08.988708 3948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:08.991228 3948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31: Bootstrap starting.
I20260810 06:49:08.991791 3948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:08.992044 3948 log.cc:824] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:08.992730 3948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31: No bootstrap required, opened a new log
I20260810 06:49:08.994459 3948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017b70c94586408a9c593aa816f31c31" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41039 } }
I20260810 06:49:08.994614 3948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:08.994637 3948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017b70c94586408a9c593aa816f31c31, State: Initialized, Role: FOLLOWER
I20260810 06:49:08.994748 3948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017b70c94586408a9c593aa816f31c31" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41039 } }
I20260810 06:49:08.994849 3948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:08.994899 3948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:08.994949 3948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:08.995507 3948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017b70c94586408a9c593aa816f31c31" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41039 } }
I20260810 06:49:08.995635 3948 leader_election.cc:302] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 017b70c94586408a9c593aa816f31c31; no voters:
I20260810 06:49:08.995834 3948 leader_election.cc:288] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:08.995885 3952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:08.996112 3948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:08.996115 3952 raft_consensus.cc:686] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [term 1 LEADER]: Becoming Leader. State: Replica: 017b70c94586408a9c593aa816f31c31, State: Running, Role: LEADER
I20260810 06:49:08.996276 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 3875
I20260810 06:49:08.996431 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:08.996412 3952 consensus_queue.cc:237] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017b70c94586408a9c593aa816f31c31" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41039 } }
I20260810 06:49:08.996645 16291 external_mini_cluster.cc:1428] Setting key 294768ac6b8adbcabb22cc47fe669de8
I20260810 06:49:08.996945 3954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 017b70c94586408a9c593aa816f31c31. Latest consensus state: current_term: 1 leader_uuid: "017b70c94586408a9c593aa816f31c31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017b70c94586408a9c593aa816f31c31" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41039 } } }
I20260810 06:49:08.997047 3954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:08.997407 3953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "017b70c94586408a9c593aa816f31c31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017b70c94586408a9c593aa816f31c31" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 41039 } } }
I20260810 06:49:08.997471 3963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:08.997493 3953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:08.997931 3963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:08.999817 3963 catalog_manager.cc:1347] Generated new cluster ID: 7bc909cdd322417f80bc15a7edfd58bf
I20260810 06:49:08.999873 3963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:09.004129 3966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:08.997434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36984 (local address 127.15.232.254:41039)
0810 06:49:08.997675 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:08.997687 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:08.998575 (+ 888us) server_negotiation.cc:407] Connection header received
0810 06:49:08.998731 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:08.998739 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:08.998841 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:08.999031 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:08.999213 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.000577 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.001786 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.002411 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.002532 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.003349 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:09.003353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:09.003353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:09.003586 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:09.003629 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:09.003644 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:09.003710 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.003816 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.003898 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":77,"threads_started":1}
I20260810 06:49:09.015035 3963 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:09.015695 3963 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:09.021096 3963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31: Generated new TSK 0
I20260810 06:49:09.021365 3963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:09.028661 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41039
--builtin_ntp_servers=127.15.232.212:39937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:49:09.118760 3976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:09.118992 3976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:09.119016 3976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:09.120225 3976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:09.120270 3976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:09.120296 3976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:49:09.121697 3976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:39937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:09.121987 3976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:09.122318 3976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:09.127815 3986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:09.127916 3983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:09.127919 3984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:09.128252 3976 server_base.cc:1034] running on GCE node
I20260810 06:49:09.128489 3976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:09.128793 3976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:09.130000 3976 hybrid_clock.cc:648] HybridClock initialized: now 1786344549129986 us; error 30 us; skew 500 ppm
I20260810 06:49:09.131508 3976 webserver.cc:460] Webserver started at http://127.15.232.193:35783/ using document root <none> and password file <none>
I20260810 06:49:09.131778 3976 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:09.131855 3976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:09.132014 3976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:09.133137 3976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "3ac30986bbac4339b5d3bf00412e6717"
format_stamp: "Formatted at 2026-08-10 06:49:09 on dist-test-slave-vxj2"
server_key: "bd56b6596172d3d64e8c744cc45b458a"
server_key_iv: "3e8461ef53db60e254a7cea94cdbf3b5"
server_key_version: "encryptionkey@0"
I20260810 06:49:09.133607 3976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "3ac30986bbac4339b5d3bf00412e6717"
format_stamp: "Formatted at 2026-08-10 06:49:09 on dist-test-slave-vxj2"
server_key: "bd56b6596172d3d64e8c744cc45b458a"
server_key_iv: "3e8461ef53db60e254a7cea94cdbf3b5"
server_key_version: "encryptionkey@0"
I20260810 06:49:09.135162 3976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:49:09.136063 3992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:09.136284 3976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:49:09.136408 3976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "3ac30986bbac4339b5d3bf00412e6717"
format_stamp: "Formatted at 2026-08-10 06:49:09 on dist-test-slave-vxj2"
server_key: "bd56b6596172d3d64e8c744cc45b458a"
server_key_iv: "3e8461ef53db60e254a7cea94cdbf3b5"
server_key_version: "encryptionkey@0"
I20260810 06:49:09.136500 3976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:09.160398 3976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:09.160781 3976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:09.160950 3976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:09.161274 3976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:09.161646 3976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:09.161702 3976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:09.161743 3976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:09.161772 3976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:09.168215 3976 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:46607
I20260810 06:49:09.168272 4120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:46607 every 8 connection(s)
I20260810 06:49:09.168694 3976 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:49:09.174724 3966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.169593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:43057 (local address 127.15.232.254:41039)
0810 06:49:09.169665 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:09.169667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:09.169892 (+ 225us) server_negotiation.cc:407] Connection header received
0810 06:49:09.170470 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:09.170471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:09.170488 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:09.170565 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:09.171303 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.171805 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.172909 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.173109 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.173365 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.174437 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:09.174439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:09.174439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:09.174449 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:09.174467 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:09.174471 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:09.174538 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.174613 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.174647 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:49:09.174937 4124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.169506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41039 (local address 127.15.232.193:43057)
0810 06:49:09.169736 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:09.169785 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:09.170335 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:09.170646 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:09.170649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:09.170658 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:09.171189 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.171193 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.171955 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.171956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.172796 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.172799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.173418 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.173419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.173548 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.174240 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:49:09.174241 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:09.174273 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:49:09.174316 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:09.174538 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:09.174541 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:09.174543 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:49:09.174592 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 06:49:09.174680 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260810 06:49:09.175243 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 3976
I20260810 06:49:09.175400 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:49:09.175634 4121 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41039
I20260810 06:49:09.175678 16291 external_mini_cluster.cc:1428] Setting key 977c9c734b58f9fc64a65e66ee716fa0
I20260810 06:49:09.175757 4121 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:09.176131 4121 heartbeater.cc:511] Master 127.15.232.254:41039 requested a full tablet report, sending...
I20260810 06:49:09.176772 3908 ts_manager.cc:194] Registered new tserver with Master: 3ac30986bbac4339b5d3bf00412e6717 (127.15.232.193:46607)
I20260810 06:49:09.176817 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41039
--builtin_ntp_servers=127.15.232.212:39937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:49:09.177753 3908 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.193:43057
W20260810 06:49:09.265331 4125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:09.265583 4125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:09.265609 4125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:09.266805 4125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:09.266849 4125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:09.266875 4125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:49:09.268154 4125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:39937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:09.268342 4125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:09.268589 4125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:49:09.273980 4125 server_base.cc:1034] running on GCE node
W20260810 06:49:09.273936 4131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:09.273947 4136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:09.273936 4133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:09.274394 4125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:09.274619 4125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:09.275779 4125 hybrid_clock.cc:648] HybridClock initialized: now 1786344549275755 us; error 39 us; skew 500 ppm
I20260810 06:49:09.277333 4125 webserver.cc:460] Webserver started at http://127.15.232.194:36783/ using document root <none> and password file <none>
I20260810 06:49:09.277603 4125 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:09.277655 4125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:09.277845 4125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:09.278993 4125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "632639fe0b4f49889e279159cc94309e"
format_stamp: "Formatted at 2026-08-10 06:49:09 on dist-test-slave-vxj2"
server_key: "e7f8a7b909ed3f23e278337acaeaaa60"
server_key_iv: "9aa0c836d0ff1cdd8d0a66e81d843272"
server_key_version: "encryptionkey@0"
I20260810 06:49:09.279366 4125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "632639fe0b4f49889e279159cc94309e"
format_stamp: "Formatted at 2026-08-10 06:49:09 on dist-test-slave-vxj2"
server_key: "e7f8a7b909ed3f23e278337acaeaaa60"
server_key_iv: "9aa0c836d0ff1cdd8d0a66e81d843272"
server_key_version: "encryptionkey@0"
I20260810 06:49:09.280755 4125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:09.281584 4142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:09.281802 4125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:49:09.281898 4125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "632639fe0b4f49889e279159cc94309e"
format_stamp: "Formatted at 2026-08-10 06:49:09 on dist-test-slave-vxj2"
server_key: "e7f8a7b909ed3f23e278337acaeaaa60"
server_key_iv: "9aa0c836d0ff1cdd8d0a66e81d843272"
server_key_version: "encryptionkey@0"
I20260810 06:49:09.281956 4125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:09.305555 4125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:09.305876 4125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:09.305991 4125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:09.306213 4125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:09.306607 4125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:09.306663 4125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:09.306722 4125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:09.306766 4125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:09.312779 4125 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:44103
I20260810 06:49:09.312855 4276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:44103 every 8 connection(s)
I20260810 06:49:09.313273 4125 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:49:09.318915 3966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.314063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:49205 (local address 127.15.232.254:41039)
0810 06:49:09.314140 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:09.314142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:09.314377 (+ 235us) server_negotiation.cc:407] Connection header received
0810 06:49:09.314870 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:09.314871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:09.314892 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:09.314958 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:09.315678 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.316187 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.317488 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.317679 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.317738 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.318658 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:09.318659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:09.318660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:09.318665 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:09.318678 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:09.318682 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:09.318727 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.318818 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.318846 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:49:09.319140 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.314014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41039 (local address 127.15.232.194:49205)
0810 06:49:09.314210 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:09.314263 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:09.314768 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:09.315035 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:09.315039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:09.315047 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:09.315592 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.315597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.316312 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.316313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.317392 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.317396 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.317787 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.317788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.317919 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.318493 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:49:09.318493 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:09.318533 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:49:09.318563 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:09.318742 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:09.318746 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:09.318749 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:49:09.318809 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 06:49:09.318905 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260810 06:49:09.319751 4277 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41039
I20260810 06:49:09.319864 4277 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:09.320071 4277 heartbeater.cc:511] Master 127.15.232.254:41039 requested a full tablet report, sending...
I20260810 06:49:09.320521 3908 ts_manager.cc:194] Registered new tserver with Master: 632639fe0b4f49889e279159cc94309e (127.15.232.194:44103)
I20260810 06:49:09.320945 3908 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.194:49205
I20260810 06:49:09.322921 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 4125
I20260810 06:49:09.323053 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:49:09.323302 16291 external_mini_cluster.cc:1428] Setting key cdd28d9323c71509c8521950e0c0804a
I20260810 06:49:09.324337 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41039
--builtin_ntp_servers=127.15.232.212:39937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:49:09.420341 4280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:09.420557 4280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:09.420580 4280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:09.421842 4280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:09.421893 4280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:09.421921 4280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:49:09.423799 4280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:39937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:41039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:09.424103 4280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:09.424437 4280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:09.429771 4287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:09.429762 4289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:09.429762 4285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:09.430011 4280 server_base.cc:1034] running on GCE node
I20260810 06:49:09.430280 4280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:09.430500 4280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:09.431663 4280 hybrid_clock.cc:648] HybridClock initialized: now 1786344549431647 us; error 29 us; skew 500 ppm
I20260810 06:49:09.433058 4280 webserver.cc:460] Webserver started at http://127.15.232.195:37675/ using document root <none> and password file <none>
I20260810 06:49:09.433363 4280 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:09.433414 4280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:09.433552 4280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:09.434664 4280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "9fccaf09d13b46fa804a6f0c2705a5d4"
format_stamp: "Formatted at 2026-08-10 06:49:09 on dist-test-slave-vxj2"
server_key: "7513d477327ab2cf416ec4ca26a7d0a0"
server_key_iv: "f96fd791c2714a6a98de4d158c63159f"
server_key_version: "encryptionkey@0"
I20260810 06:49:09.435042 4280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "9fccaf09d13b46fa804a6f0c2705a5d4"
format_stamp: "Formatted at 2026-08-10 06:49:09 on dist-test-slave-vxj2"
server_key: "7513d477327ab2cf416ec4ca26a7d0a0"
server_key_iv: "f96fd791c2714a6a98de4d158c63159f"
server_key_version: "encryptionkey@0"
I20260810 06:49:09.436448 4280 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:09.437399 4299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:09.437629 4280 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:09.437805 4280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "9fccaf09d13b46fa804a6f0c2705a5d4"
format_stamp: "Formatted at 2026-08-10 06:49:09 on dist-test-slave-vxj2"
server_key: "7513d477327ab2cf416ec4ca26a7d0a0"
server_key_iv: "f96fd791c2714a6a98de4d158c63159f"
server_key_version: "encryptionkey@0"
I20260810 06:49:09.437879 4280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:09.445982 4280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:09.446321 4280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:09.446496 4280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:09.446807 4280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:09.447175 4280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:09.447235 4280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:09.447283 4280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:09.447307 4280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:09.453938 4280 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:35777
I20260810 06:49:09.453984 4434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:35777 every 8 connection(s)
I20260810 06:49:09.454350 4280 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:49:09.459937 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 4280
I20260810 06:49:09.460065 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:49:09.460246 3966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.455174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:51109 (local address 127.15.232.254:41039)
0810 06:49:09.455254 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:09.455257 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:09.455543 (+ 286us) server_negotiation.cc:407] Connection header received
0810 06:49:09.456063 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:09.456064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:09.456075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:09.456138 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:09.456911 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.457542 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.458560 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.458818 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.458881 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.459869 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:09.459870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:09.459871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:09.459878 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:09.459896 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:09.459902 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:09.459981 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.460113 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.460151 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:49:09.460348 16291 external_mini_cluster.cc:1428] Setting key 5f39fe5d185098e56b44eee00c8dfa8a
I20260810 06:49:09.460459 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.455157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:41039 (local address 127.15.232.195:51109)
0810 06:49:09.455417 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:09.455453 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:09.455950 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:09.456218 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:09.456222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:09.456230 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:09.456789 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.456794 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.457635 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.457636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.458465 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.458468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.458901 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.458902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.459044 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.459696 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:49:09.459697 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:09.459728 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:49:09.459756 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:09.459988 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:09.459992 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:09.459995 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:49:09.460090 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 06:49:09.460213 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":114,"threads_started":1}
I20260810 06:49:09.460947 4436 heartbeater.cc:347] Connected to a master server at 127.15.232.254:41039
I20260810 06:49:09.461064 4436 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:09.461295 4436 heartbeater.cc:511] Master 127.15.232.254:41039 requested a full tablet report, sending...
I20260810 06:49:09.461772 3907 ts_manager.cc:194] Registered new tserver with Master: 9fccaf09d13b46fa804a6f0c2705a5d4 (127.15.232.195:35777)
I20260810 06:49:09.462182 3907 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.195:51109
I20260810 06:49:09.472512 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:49:09.477566 3966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.474212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36996 (local address 127.15.232.254:41039)
0810 06:49:09.474279 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:09.474280 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:09.474387 (+ 107us) server_negotiation.cc:407] Connection header received
0810 06:49:09.474447 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:09.474447 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:09.474459 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:09.474517 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:09.474721 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.475242 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.476067 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.476272 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.476332 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.477034 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:09.477035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:09.477036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:09.477040 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:09.477054 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:09.477058 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:09.477120 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.477467 (+ 347us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.477496 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 06:49:09.479532 3907 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:49:09.481133 3907 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:09.488361 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.483904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43504 (local address 127.15.232.193:46607)
0810 06:49:09.484204 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:09.484208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:09.484217 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:49:09.484252 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:09.484255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:09.484268 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:09.484347 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:09.484587 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.485877 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.487129 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.487896 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.487983 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.488082 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.488196 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.488239 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
I20260810 06:49:09.488351 4446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.483667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:44103 (local address 127.0.0.1:57318)
0810 06:49:09.483847 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:09.483990 (+ 143us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:09.484061 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:09.484342 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:09.484346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:09.484348 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:09.484427 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.484429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.485607 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.485608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.487022 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.487026 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.487994 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.487995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.488088 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.488092 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:49:09.488167 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:49:09.488245 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":28,"thread_start_us":67,"threads_started":1}
I20260810 06:49:09.488354 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.483683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:46607 (local address 127.0.0.1:43504)
0810 06:49:09.484010 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:09.484022 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:09.484079 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:09.484436 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:09.484437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:09.484440 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:09.484503 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.484505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.485980 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.485981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.487025 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.487027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.487977 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.487978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.488073 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.488089 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 06:49:09.488168 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:49:09.488237 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":7,"thread_start_us":76,"threads_started":1}
I20260810 06:49:09.488945 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.483776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57318 (local address 127.15.232.194:44103)
0810 06:49:09.484149 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:09.484155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:09.484166 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:49:09.484210 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:09.484212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:09.484230 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:09.484318 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:09.484525 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.485518 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.487130 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.487901 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.488580 (+ 679us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.488704 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.488734 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.488840 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":101,"threads_started":1}
I20260810 06:49:09.489144 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.483995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.15.232.195:35777)
0810 06:49:09.484200 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:09.484204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:09.484380 (+ 176us) server_negotiation.cc:407] Connection header received
0810 06:49:09.484423 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:09.484425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:09.484442 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:09.484516 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:09.484743 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.485891 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.487907 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.488662 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.488731 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.488824 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.489030 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.489075 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":80,"threads_started":1}
I20260810 06:49:09.489351 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.483892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:35777 (local address 127.0.0.1:54686)
0810 06:49:09.484246 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:09.484254 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:09.484326 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:09.484595 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:09.484596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:09.484598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:09.484673 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.484674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.487135 (+ 2461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.487136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.487809 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.487811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.488868 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.488869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.488934 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.488938 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:49:09.488997 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 06:49:09.489285 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":64,"threads_started":1}
I20260810 06:49:09.489707 4044 tablet_service.cc:1467] Processing CreateTablet for tablet 618e82c1dd274e4a8686ec0f7e32e461 (DEFAULT_TABLE table=test-table [id=f08c222e868142238db01970b71a3cc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:09.489708 4195 tablet_service.cc:1467] Processing CreateTablet for tablet 618e82c1dd274e4a8686ec0f7e32e461 (DEFAULT_TABLE table=test-table [id=f08c222e868142238db01970b71a3cc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:09.490038 4044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 618e82c1dd274e4a8686ec0f7e32e461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:09.490077 4195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 618e82c1dd274e4a8686ec0f7e32e461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:09.490056 4356 tablet_service.cc:1467] Processing CreateTablet for tablet 618e82c1dd274e4a8686ec0f7e32e461 (DEFAULT_TABLE table=test-table [id=f08c222e868142238db01970b71a3cc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:09.490370 4356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 618e82c1dd274e4a8686ec0f7e32e461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:09.492988 4452 tablet_bootstrap.cc:492] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717: Bootstrap starting.
I20260810 06:49:09.493553 4453 tablet_bootstrap.cc:492] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e: Bootstrap starting.
I20260810 06:49:09.493789 4456 tablet_bootstrap.cc:492] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4: Bootstrap starting.
I20260810 06:49:09.493950 4452 tablet_bootstrap.cc:654] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:09.494210 4453 tablet_bootstrap.cc:654] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:09.494298 4452 log.cc:824] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:09.494422 4456 tablet_bootstrap.cc:654] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:09.494551 4453 log.cc:824] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:09.494750 4456 log.cc:824] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:09.495141 4452 tablet_bootstrap.cc:492] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717: No bootstrap required, opened a new log
I20260810 06:49:09.495224 4452 ts_tablet_manager.cc:1397] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 06:49:09.495471 4456 tablet_bootstrap.cc:492] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4: No bootstrap required, opened a new log
I20260810 06:49:09.495539 4456 ts_tablet_manager.cc:1397] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:49:09.495599 4453 tablet_bootstrap.cc:492] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e: No bootstrap required, opened a new log
I20260810 06:49:09.495673 4453 ts_tablet_manager.cc:1397] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:49:09.497928 4453 raft_consensus.cc:348] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632639fe0b4f49889e279159cc94309e" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44103 } } peers { permanent_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 35777 } } peers { permanent_uuid: "3ac30986bbac4339b5d3bf00412e6717" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46607 } }
I20260810 06:49:09.498114 4453 raft_consensus.cc:374] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:09.498162 4453 raft_consensus.cc:729] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632639fe0b4f49889e279159cc94309e, State: Initialized, Role: FOLLOWER
I20260810 06:49:09.498313 4453 consensus_queue.cc:260] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632639fe0b4f49889e279159cc94309e" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44103 } } peers { permanent_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 35777 } } peers { permanent_uuid: "3ac30986bbac4339b5d3bf00412e6717" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46607 } }
I20260810 06:49:09.498384 4456 raft_consensus.cc:348] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632639fe0b4f49889e279159cc94309e" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44103 } } peers { permanent_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 35777 } } peers { permanent_uuid: "3ac30986bbac4339b5d3bf00412e6717" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46607 } }
I20260810 06:49:09.498389 4452 raft_consensus.cc:348] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632639fe0b4f49889e279159cc94309e" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44103 } } peers { permanent_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 35777 } } peers { permanent_uuid: "3ac30986bbac4339b5d3bf00412e6717" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46607 } }
I20260810 06:49:09.498540 4452 raft_consensus.cc:374] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:09.498540 4456 raft_consensus.cc:374] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:09.498575 4456 raft_consensus.cc:729] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fccaf09d13b46fa804a6f0c2705a5d4, State: Initialized, Role: FOLLOWER
I20260810 06:49:09.498575 4452 raft_consensus.cc:729] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac30986bbac4339b5d3bf00412e6717, State: Initialized, Role: FOLLOWER
I20260810 06:49:09.498631 4453 ts_tablet_manager.cc:1428] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:49:09.498697 4456 consensus_queue.cc:260] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632639fe0b4f49889e279159cc94309e" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44103 } } peers { permanent_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 35777 } } peers { permanent_uuid: "3ac30986bbac4339b5d3bf00412e6717" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46607 } }
I20260810 06:49:09.498750 4277 heartbeater.cc:503] Master 127.15.232.254:41039 was elected leader, sending a full tablet report...
I20260810 06:49:09.498770 4452 consensus_queue.cc:260] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632639fe0b4f49889e279159cc94309e" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44103 } } peers { permanent_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 35777 } } peers { permanent_uuid: "3ac30986bbac4339b5d3bf00412e6717" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46607 } }
I20260810 06:49:09.498979 4456 ts_tablet_manager.cc:1428] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 06:49:09.499009 4452 ts_tablet_manager.cc:1428] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260810 06:49:09.499086 4464 raft_consensus.cc:482] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:09.499112 4121 heartbeater.cc:503] Master 127.15.232.254:41039 was elected leader, sending a full tablet report...
I20260810 06:49:09.499197 4436 heartbeater.cc:503] Master 127.15.232.254:41039 was elected leader, sending a full tablet report...
I20260810 06:49:09.499738 4464 raft_consensus.cc:504] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632639fe0b4f49889e279159cc94309e" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44103 } } peers { permanent_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 35777 } } peers { permanent_uuid: "3ac30986bbac4339b5d3bf00412e6717" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46607 } }
I20260810 06:49:09.500150 4464 leader_election.cc:288] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632639fe0b4f49889e279159cc94309e (127.15.232.194:44103), 9fccaf09d13b46fa804a6f0c2705a5d4 (127.15.232.195:35777)
I20260810 06:49:09.503648 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.500678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:42341 (local address 127.15.232.195:35777)
0810 06:49:09.500785 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:09.500788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:09.500796 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:49:09.500821 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:09.500822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:09.500831 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:09.500887 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:09.501101 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.501825 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.502765 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.503294 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.503364 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.503373 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.503538 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.503576 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 06:49:09.503858 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.500438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:35777 (local address 127.15.232.193:42341)
0810 06:49:09.500585 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:09.500625 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:09.500667 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:09.500935 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:09.500936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:09.500938 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:09.501020 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.501021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.501913 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.501914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.502665 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.502667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.503375 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.503377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.503452 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.503456 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:49:09.503736 (+ 280us) client_negotiation.cc:240] Negotiation successful
0810 06:49:09.503780 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260810 06:49:09.504554 4377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "618e82c1dd274e4a8686ec0f7e32e461" candidate_uuid: "3ac30986bbac4339b5d3bf00412e6717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4" is_pre_election: true
I20260810 06:49:09.504657 4124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.500329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:44103 (local address 127.15.232.193:59819)
0810 06:49:09.500375 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:09.500389 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:09.500429 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:09.500814 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:09.500815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:09.500816 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:09.500886 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.500888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.501868 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.501869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.502656 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:09.502658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.504422 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.504423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:09.504494 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.504497 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:49:09.504554 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 06:49:09.504594 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":30}
I20260810 06:49:09.504734 4377 raft_consensus.cc:2454] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ac30986bbac4339b5d3bf00412e6717 in term 0.
I20260810 06:49:09.505007 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.500658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:59819 (local address 127.15.232.194:44103)
0810 06:49:09.500704 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:09.500706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:09.500714 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:49:09.500735 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:09.500736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:09.500745 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:09.500810 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:09.500949 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.501731 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.503408 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.503974 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.504049 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.504055 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.504903 (+ 848us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.504948 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:49:09.504987 3995 leader_election.cc:302] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ac30986bbac4339b5d3bf00412e6717, 9fccaf09d13b46fa804a6f0c2705a5d4; no voters:
I20260810 06:49:09.505127 4464 raft_consensus.cc:2781] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:09.505214 4464 raft_consensus.cc:482] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:09.505249 4464 raft_consensus.cc:3037] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:09.506038 4464 raft_consensus.cc:504] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632639fe0b4f49889e279159cc94309e" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44103 } } peers { permanent_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 35777 } } peers { permanent_uuid: "3ac30986bbac4339b5d3bf00412e6717" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46607 } }
I20260810 06:49:09.506255 4220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "618e82c1dd274e4a8686ec0f7e32e461" candidate_uuid: "3ac30986bbac4339b5d3bf00412e6717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632639fe0b4f49889e279159cc94309e" is_pre_election: true
I20260810 06:49:09.506389 4219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "618e82c1dd274e4a8686ec0f7e32e461" candidate_uuid: "3ac30986bbac4339b5d3bf00412e6717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632639fe0b4f49889e279159cc94309e"
I20260810 06:49:09.506428 4220 raft_consensus.cc:2454] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ac30986bbac4339b5d3bf00412e6717 in term 0.
I20260810 06:49:09.506369 4377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "618e82c1dd274e4a8686ec0f7e32e461" candidate_uuid: "3ac30986bbac4339b5d3bf00412e6717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4"
I20260810 06:49:09.506605 4377 raft_consensus.cc:3037] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:09.506635 4464 leader_election.cc:288] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [CANDIDATE]: Term 1 election: Requested vote from peers 632639fe0b4f49889e279159cc94309e (127.15.232.194:44103), 9fccaf09d13b46fa804a6f0c2705a5d4 (127.15.232.195:35777)
I20260810 06:49:09.507388 4377 raft_consensus.cc:2454] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ac30986bbac4339b5d3bf00412e6717 in term 1.
I20260810 06:49:09.507601 3995 leader_election.cc:302] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ac30986bbac4339b5d3bf00412e6717, 9fccaf09d13b46fa804a6f0c2705a5d4; no voters: 632639fe0b4f49889e279159cc94309e
I20260810 06:49:09.507710 4464 raft_consensus.cc:2781] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:09.507793 4464 raft_consensus.cc:686] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [term 1 LEADER]: Becoming Leader. State: Replica: 3ac30986bbac4339b5d3bf00412e6717, State: Running, Role: LEADER
I20260810 06:49:09.507881 4464 consensus_queue.cc:237] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632639fe0b4f49889e279159cc94309e" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44103 } } peers { permanent_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 35777 } } peers { permanent_uuid: "3ac30986bbac4339b5d3bf00412e6717" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46607 } }
I20260810 06:49:09.508845 3906 catalog_manager.cc:5495] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ac30986bbac4339b5d3bf00412e6717 (127.15.232.193). New cstate: current_term: 1 leader_uuid: "3ac30986bbac4339b5d3bf00412e6717" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "632639fe0b4f49889e279159cc94309e" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 35777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ac30986bbac4339b5d3bf00412e6717" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 46607 } health_report { overall_health: HEALTHY } } }
I20260810 06:49:09.517537 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.514961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.15.232.193:46607)
0810 06:49:09.515029 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:09.515032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:09.515040 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:49:09.515115 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:09.515116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:09.515135 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:09.515233 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:09.515391 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.515867 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.516649 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:09.516861 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:09.516936 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:09.517101 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:09.517274 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:09.517366 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.517407 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.517427 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:49:09.519434 4377 raft_consensus.cc:1261] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4 [term 1 FOLLOWER]: Refusing update from remote peer 3ac30986bbac4339b5d3bf00412e6717: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:09.519410 4219 raft_consensus.cc:3037] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:09.520277 4219 raft_consensus.cc:1261] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e [term 1 FOLLOWER]: Refusing update from remote peer 3ac30986bbac4339b5d3bf00412e6717: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:09.520609 4464 consensus_queue.cc:1035] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "632639fe0b4f49889e279159cc94309e" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:09.520744 4462 consensus_queue.cc:1035] T 618e82c1dd274e4a8686ec0f7e32e461 P 3ac30986bbac4339b5d3bf00412e6717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fccaf09d13b46fa804a6f0c2705a5d4" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 35777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:09.523963 4472 mvcc.cc:205] Tried to move back new op lower bound from 7316867274828664832 to 7316867274785030144. Current Snapshot: MvccSnapshot[applied={T|T < 7316867274828664832}]
I20260810 06:49:09.524063 4470 mvcc.cc:205] Tried to move back new op lower bound from 7316867274828664832 to 7316867274785030144. Current Snapshot: MvccSnapshot[applied={T|T < 7316867274828664832 or (T in {7316867274828664832})}]
I20260810 06:49:09.524195 4471 mvcc.cc:205] Tried to move back new op lower bound from 7316867274828664832 to 7316867274785030144. Current Snapshot: MvccSnapshot[applied={T|T < 7316867274828664832}]
I20260810 06:49:09.525683 3907 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:09.525810 3907 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:09.526698 3907 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 017b70c94586408a9c593aa816f31c31: Sending DeleteTablet for 3 replicas of tablet 618e82c1dd274e4a8686ec0f7e32e461
I20260810 06:49:09.527093 4195 tablet_service.cc:1514] Processing DeleteTablet for tablet 618e82c1dd274e4a8686ec0f7e32e461 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:09 UTC) from {username='slave'} at 127.0.0.1:57318
I20260810 06:49:09.527191 4044 tablet_service.cc:1514] Processing DeleteTablet for tablet 618e82c1dd274e4a8686ec0f7e32e461 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:09 UTC) from {username='slave'} at 127.0.0.1:43504
I20260810 06:49:09.527361 4480 tablet_replica.cc:330] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e: stopping tablet replica
I20260810 06:49:09.527473 4480 raft_consensus.cc:2229] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:09.527462 4356 tablet_service.cc:1514] Processing DeleteTablet for tablet 618e82c1dd274e4a8686ec0f7e32e461 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:09 UTC) from {username='slave'} at 127.0.0.1:54686
I20260810 06:49:09.527583 4480 raft_consensus.cc:2258] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:09.527675 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 3976
I20260810 06:49:09.528046 4480 ts_tablet_manager.cc:1905] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:09.529824 4480 ts_tablet_manager.cc:1918] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:09.529882 4480 log.cc:1196] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/618e82c1dd274e4a8686ec0f7e32e461
I20260810 06:49:09.530079 4480 ts_tablet_manager.cc:1939] T 618e82c1dd274e4a8686ec0f7e32e461 P 632639fe0b4f49889e279159cc94309e: Deleting consensus metadata
I20260810 06:49:09.531002 3893 catalog_manager.cc:4841] TS 632639fe0b4f49889e279159cc94309e (127.15.232.194:44103): tablet 618e82c1dd274e4a8686ec0f7e32e461 (table test-table [id=f08c222e868142238db01970b71a3cc5]) successfully deleted
I20260810 06:49:09.531728 4482 tablet_replica.cc:330] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4: stopping tablet replica
I20260810 06:49:09.531896 4482 raft_consensus.cc:2229] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:09.532027 4482 raft_consensus.cc:2258] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:09.532541 4482 ts_tablet_manager.cc:1905] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:09.534173 4482 ts_tablet_manager.cc:1918] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:09.534291 4482 log.cc:1196] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/618e82c1dd274e4a8686ec0f7e32e461
I20260810 06:49:09.534578 4482 ts_tablet_manager.cc:1939] T 618e82c1dd274e4a8686ec0f7e32e461 P 9fccaf09d13b46fa804a6f0c2705a5d4: Deleting consensus metadata
I20260810 06:49:09.535054 3893 catalog_manager.cc:4841] TS 9fccaf09d13b46fa804a6f0c2705a5d4 (127.15.232.195:35777): tablet 618e82c1dd274e4a8686ec0f7e32e461 (table test-table [id=f08c222e868142238db01970b71a3cc5]) successfully deleted
W20260810 06:49:09.536784 3894 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:46607 (error 108)
I20260810 06:49:09.537062 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 4125
I20260810 06:49:09.538472 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.537259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:46607 (local address 127.0.0.1:43512)
0810 06:49:09.538368 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:09.538425 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:46607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1078}
W20260810 06:49:09.538785 3894 catalog_manager.cc:4584] TS 3ac30986bbac4339b5d3bf00412e6717 (127.15.232.193:46607): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 618e82c1dd274e4a8686ec0f7e32e461: Network error: Client connection negotiation failed: client connection to 127.15.232.193:46607: connect: Connection refused (error 111)
I20260810 06:49:09.543609 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 4280
I20260810 06:49:09.551631 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 3875
2026-08-10T06:49:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (720 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:09 dist-test-slave-vxj2 krb5kdc[4487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:09 dist-test-slave-vxj2 krb5kdc[4487](info): set up 2 sockets
Aug 10 06:49:09 dist-test-slave-vxj2 krb5kdc[4487](info): commencing operation
krb5kdc: starting...
W20260810 06:49:10.684852 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:10 dist-test-slave-vxj2 krb5kdc[4487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:49:10.817592 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:44785
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:44785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:10.912088 4501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:10.912395 4501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:10.912423 4501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:10.913936 4501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:10.913991 4501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:10.914007 4501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:10.914021 4501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:10.914036 4501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:10.915831 4501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:44785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:44785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:10.916152 4501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:10.916435 4501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:10.922238 4507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:10.922190 4508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:10.922204 4510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:10.922463 4501 server_base.cc:1034] running on GCE node
I20260810 06:49:10.922717 4501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:10.923058 4501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:10.924219 4501 hybrid_clock.cc:648] HybridClock initialized: now 1786344550924192 us; error 38 us; skew 500 ppm
Aug 10 06:49:10 dist-test-slave-vxj2 krb5kdc[4487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344550, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:10.927222 4501 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:49:10.927774 4501 webserver.cc:460] Webserver started at http://127.15.232.254:37765/ using document root <none> and password file <none>
I20260810 06:49:10.928049 4501 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:10.928123 4501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:10.928278 4501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:10.929497 4501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "b3941c0ef87145f4bd7054057a4964fe"
format_stamp: "Formatted at 2026-08-10 06:49:10 on dist-test-slave-vxj2"
server_key: "c2607d0176ea4b307dbbfaca9653934f"
server_key_iv: "8bf448560f5b6dfe8f27ed44a14accdd"
server_key_version: "encryptionkey@0"
I20260810 06:49:10.929888 4501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "b3941c0ef87145f4bd7054057a4964fe"
format_stamp: "Formatted at 2026-08-10 06:49:10 on dist-test-slave-vxj2"
server_key: "c2607d0176ea4b307dbbfaca9653934f"
server_key_iv: "8bf448560f5b6dfe8f27ed44a14accdd"
server_key_version: "encryptionkey@0"
I20260810 06:49:10.931370 4501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:49:10.932225 4517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:10.932425 4501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:10.932538 4501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "b3941c0ef87145f4bd7054057a4964fe"
format_stamp: "Formatted at 2026-08-10 06:49:10 on dist-test-slave-vxj2"
server_key: "c2607d0176ea4b307dbbfaca9653934f"
server_key_iv: "8bf448560f5b6dfe8f27ed44a14accdd"
server_key_version: "encryptionkey@0"
I20260810 06:49:10.932619 4501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:10.942533 4501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:10.945034 4501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:10.945248 4501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:10.949606 4501 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:44785
I20260810 06:49:10.949671 4577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:44785 every 8 connection(s)
I20260810 06:49:10.950063 4501 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:10.951074 4579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:10.953668 4579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe: Bootstrap starting.
I20260810 06:49:10.953949 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 4501
I20260810 06:49:10.954105 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:10.954360 16291 external_mini_cluster.cc:1428] Setting key e84a572b5cc0611a5791d0e0bc79b965
I20260810 06:49:10.954559 4579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:10.954967 4579 log.cc:824] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:10.955838 4579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe: No bootstrap required, opened a new log
I20260810 06:49:10.958562 4579 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3941c0ef87145f4bd7054057a4964fe" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44785 } }
I20260810 06:49:10.958774 4579 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:10.958814 4579 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3941c0ef87145f4bd7054057a4964fe, State: Initialized, Role: FOLLOWER
I20260810 06:49:10.958959 4579 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3941c0ef87145f4bd7054057a4964fe" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44785 } }
I20260810 06:49:10.959071 4579 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:10.959152 4579 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:10.959210 4579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:10.960095 4579 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3941c0ef87145f4bd7054057a4964fe" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44785 } }
I20260810 06:49:10.960199 4579 leader_election.cc:302] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3941c0ef87145f4bd7054057a4964fe; no voters:
I20260810 06:49:10.960415 4579 leader_election.cc:288] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:10.960539 4584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:10.960708 4584 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [term 1 LEADER]: Becoming Leader. State: Replica: b3941c0ef87145f4bd7054057a4964fe, State: Running, Role: LEADER
I20260810 06:49:10.960763 4579 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:10.960810 4584 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3941c0ef87145f4bd7054057a4964fe" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44785 } }
I20260810 06:49:10.961233 4584 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3941c0ef87145f4bd7054057a4964fe. Latest consensus state: current_term: 1 leader_uuid: "b3941c0ef87145f4bd7054057a4964fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3941c0ef87145f4bd7054057a4964fe" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44785 } } }
I20260810 06:49:10.961345 4584 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:10.961252 4585 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3941c0ef87145f4bd7054057a4964fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3941c0ef87145f4bd7054057a4964fe" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44785 } } }
I20260810 06:49:10.961560 4585 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:10.961762 4591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:10.962381 4591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:49:10 dist-test-slave-vxj2 krb5kdc[4487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:10.964475 4591 catalog_manager.cc:1347] Generated new cluster ID: b7cb7b1f5eec4a5fb3896d9d83fd07ad
I20260810 06:49:10.964542 4591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:10.967168 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:10.954969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41350 (local address 127.15.232.254:44785)
0810 06:49:10.955201 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:10.955210 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:10.955236 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:49:10.955675 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:10.955682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:10.955731 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:10.955862 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:10.956780 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:10.957785 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:10.959040 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:10.959636 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:10.959724 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:10.963250 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:10.963255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:10.963256 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:10.963273 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:10.965403 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:10.965824 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:10.965828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:10.965829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:10.965929 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:10.966170 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:10.966171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:10.966171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:10.966431 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:10.966648 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:10.966728 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 06:49:10.967000 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":98,"threads_started":1}
I20260810 06:49:10.975600 4591 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:10.976449 4591 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:10.982601 4591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe: Generated new TSK 0
I20260810 06:49:10.982851 4591 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:49:11.015939 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44785
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:11.103281 4607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:11.103515 4607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:11.103541 4607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:11.104761 4607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:11.104815 4607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:11.104848 4607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:49:11.106438 4607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:11.106739 4607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:11.107084 4607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:11.112495 4615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:11.112504 4617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:11.112519 4607 server_base.cc:1034] running on GCE node
W20260810 06:49:11.112787 4619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:11.113090 4607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:11.113415 4607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:11.114586 4607 hybrid_clock.cc:648] HybridClock initialized: now 1786344551114565 us; error 40 us; skew 500 ppm
Aug 10 06:49:11 dist-test-slave-vxj2 krb5kdc[4487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:11.117540 4607 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:49:11.118021 4607 webserver.cc:460] Webserver started at http://127.15.232.193:41729/ using document root <none> and password file <none>
I20260810 06:49:11.118362 4607 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:11.118458 4607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:11.118628 4607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:11.119747 4607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "270b0fc91c5246138a920496e61994d4"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "32b4443fcb21835cb409ea929bc05531"
server_key_iv: "bf679de762b5812c1489bdf9695d379a"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.120141 4607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "270b0fc91c5246138a920496e61994d4"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "32b4443fcb21835cb409ea929bc05531"
server_key_iv: "bf679de762b5812c1489bdf9695d379a"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.121732 4607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:11.122576 4626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.122838 4607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:11.122997 4607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "270b0fc91c5246138a920496e61994d4"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "32b4443fcb21835cb409ea929bc05531"
server_key_iv: "bf679de762b5812c1489bdf9695d379a"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.123062 4607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:11.139792 4607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:11.142084 4607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:11.142217 4607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:11.142433 4607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:11.142788 4607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:11.142825 4607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.142850 4607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:11.142905 4607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.149137 4607 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:36901
I20260810 06:49:11.149240 4755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:36901 every 8 connection(s)
I20260810 06:49:11.149603 4607 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:49:11.151739 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 4607
I20260810 06:49:11.151904 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:49:11.152124 16291 external_mini_cluster.cc:1428] Setting key 189e6e15e10ba9769e23c0b8b1ea7f1b
Aug 10 06:49:11 dist-test-slave-vxj2 krb5kdc[4487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
I20260810 06:49:11.160730 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.150419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:51205 (local address 127.15.232.254:44785)
0810 06:49:11.150485 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.150487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.150703 (+ 216us) server_negotiation.cc:407] Connection header received
0810 06:49:11.151219 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.151221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.151232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.151292 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:11.152497 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.153236 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.154411 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.154647 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.154734 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.157499 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:11.157501 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:11.157501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:11.157512 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:11.159085 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:11.159569 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:11.159571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:11.159572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:11.159638 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:11.159933 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:11.159934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:11.159934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:11.160070 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:11.160152 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.160487 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.160629 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 06:49:11.160966 4758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.150385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44785 (local address 127.15.232.193:51205)
0810 06:49:11.150568 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:11.150609 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:11.151121 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:11.151380 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:11.151391 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:11.152111 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:11.152413 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.152420 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.153377 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.153378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.154259 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.154262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.154762 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.154763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.154906 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.155526 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:11.155536 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:11.157379 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:11.159204 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:11.159214 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:11.159216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:11.159480 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:11.159713 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:11.159714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:11.159714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:11.159859 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:11.160142 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:11.160150 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:11.160326 (+ 176us) client_negotiation.cc:769] Sending connection context
0810 06:49:11.160454 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 06:49:11.160650 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":114,"threads_started":1}
I20260810 06:49:11.161679 4756 heartbeater.cc:347] Connected to a master server at 127.15.232.254:44785
I20260810 06:49:11.161789 4756 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:11.162050 4756 heartbeater.cc:511] Master 127.15.232.254:44785 requested a full tablet report, sending...
I20260810 06:49:11.162717 4535 ts_manager.cc:194] Registered new tserver with Master: 270b0fc91c5246138a920496e61994d4 (127.15.232.193:36901)
I20260810 06:49:11.163479 4535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:51205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:49:11.181524 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44785
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:11.268406 4761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:11.268643 4761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:11.268671 4761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:11.269990 4761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:11.270046 4761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:11.270076 4761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:49:11.271603 4761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:11.271888 4761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:11.272140 4761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:11.277637 4767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:11.277635 4768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:11.277700 4761 server_base.cc:1034] running on GCE node
W20260810 06:49:11.277629 4771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:11.278168 4761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:11.278399 4761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:11.279548 4761 hybrid_clock.cc:648] HybridClock initialized: now 1786344551279535 us; error 30 us; skew 500 ppm
Aug 10 06:49:11 dist-test-slave-vxj2 krb5kdc[4487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:11.282330 4761 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:49:11.282956 4761 webserver.cc:460] Webserver started at http://127.15.232.194:44003/ using document root <none> and password file <none>
I20260810 06:49:11.283272 4761 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:11.283361 4761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:11.283535 4761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:11.284750 4761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "8685d11d976946e89a4781928ec66372"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "27d9d059d2340bbaea2a4acfbc3c1c1d"
server_key_iv: "0885d355c0271a5685559f9b14c6eb26"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.285230 4761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "8685d11d976946e89a4781928ec66372"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "27d9d059d2340bbaea2a4acfbc3c1c1d"
server_key_iv: "0885d355c0271a5685559f9b14c6eb26"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.286911 4761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:49:11.287870 4781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.288213 4761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:49:11.288347 4761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "8685d11d976946e89a4781928ec66372"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "27d9d059d2340bbaea2a4acfbc3c1c1d"
server_key_iv: "0885d355c0271a5685559f9b14c6eb26"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.288455 4761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:11.303953 4761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:11.306280 4761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:11.306475 4761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:11.306753 4761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:11.307101 4761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:11.307165 4761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.307201 4761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:11.307222 4761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.313483 4761 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:43603
I20260810 06:49:11.313524 4911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:43603 every 8 connection(s)
I20260810 06:49:11.313882 4761 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:49:11.317488 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 4761
I20260810 06:49:11.317593 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:49:11.317924 16291 external_mini_cluster.cc:1428] Setting key 0df3fa73f81e2190c00060e596163637
Aug 10 06:49:11 dist-test-slave-vxj2 krb5kdc[4487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:11.325083 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.314727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:43313 (local address 127.15.232.254:44785)
0810 06:49:11.314813 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.314815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.315052 (+ 237us) server_negotiation.cc:407] Connection header received
0810 06:49:11.315571 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.315572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.315583 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.315640 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:11.316663 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.317189 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.318625 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.318882 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.318974 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.321749 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:11.321750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:11.321751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:11.321758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:11.323444 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:11.323949 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:11.323950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:11.323951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:11.324019 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:11.324358 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:11.324359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:11.324360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:11.324497 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:11.324578 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.324886 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.324985 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:49:11.325346 4914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.314689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44785 (local address 127.15.232.194:43313)
0810 06:49:11.314895 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:11.314937 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:11.315459 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:11.315717 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:11.315720 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:11.316367 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:11.316573 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.316578 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.317314 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.317315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.318514 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.318517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.318988 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.318989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.319132 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.319800 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:11.319810 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:11.321592 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:11.323563 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:11.323574 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:11.323576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:11.323858 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:11.324103 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:11.324104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:11.324105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:11.324236 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:11.324568 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:11.324576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:11.324741 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 06:49:11.324865 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 06:49:11.325048 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":116,"threads_started":1}
I20260810 06:49:11.325899 4912 heartbeater.cc:347] Connected to a master server at 127.15.232.254:44785
I20260810 06:49:11.326030 4912 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:11.326304 4912 heartbeater.cc:511] Master 127.15.232.254:44785 requested a full tablet report, sending...
I20260810 06:49:11.326818 4535 ts_manager.cc:194] Registered new tserver with Master: 8685d11d976946e89a4781928ec66372 (127.15.232.194:43603)
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
I20260810 06:49:11.327314 4535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:43313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:49:11.349201 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44785
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:45681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:11.439478 4918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:11.439709 4918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:11.439734 4918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:11.440899 4918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:11.440946 4918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:11.440973 4918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:49:11.442481 4918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:11.442703 4918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:11.442919 4918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:11.448734 4925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:11.448734 4928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:11.448750 4924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:11.449316 4918 server_base.cc:1034] running on GCE node
I20260810 06:49:11.449565 4918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:11.449832 4918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:11.450975 4918 hybrid_clock.cc:648] HybridClock initialized: now 1786344551450964 us; error 32 us; skew 500 ppm
Aug 10 06:49:11 dist-test-slave-vxj2 krb5kdc[4487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:11.457346 4918 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:49:11.457870 4918 webserver.cc:460] Webserver started at http://127.15.232.195:32831/ using document root <none> and password file <none>
I20260810 06:49:11.458076 4918 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:11.458120 4918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:11.458218 4918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:11.459272 4918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "7f7b615dd2814c9da40be4052e6ebd6a"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "79f48389a3391c244e4faa4a5158ac59"
server_key_iv: "fb36e189b1ac5611d8ae77c8f60961f4"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.459591 4918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "7f7b615dd2814c9da40be4052e6ebd6a"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "79f48389a3391c244e4faa4a5158ac59"
server_key_iv: "fb36e189b1ac5611d8ae77c8f60961f4"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.461089 4918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:11.462149 4936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.462390 4918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:49:11.462530 4918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "7f7b615dd2814c9da40be4052e6ebd6a"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "79f48389a3391c244e4faa4a5158ac59"
server_key_iv: "fb36e189b1ac5611d8ae77c8f60961f4"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.462637 4918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:11.480161 4918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:11.483047 4918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:11.483255 4918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:11.483536 4918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:11.483894 4918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:11.483954 4918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.484004 4918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:11.484027 4918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.490442 4918 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:45173
I20260810 06:49:11.490517 5066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:45173 every 8 connection(s)
I20260810 06:49:11.490877 4918 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:49:11.495276 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 4918
I20260810 06:49:11.495405 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:49:11.495580 16291 external_mini_cluster.cc:1428] Setting key 53dea9a38913360e646580607b728673
Aug 10 06:49:11 dist-test-slave-vxj2 krb5kdc[4487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:11.500927 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.491710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:57785 (local address 127.15.232.254:44785)
0810 06:49:11.491801 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.491803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.492038 (+ 235us) server_negotiation.cc:407] Connection header received
0810 06:49:11.492608 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.492609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.492624 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.492676 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:11.493756 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.494283 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.495318 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.495620 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.495685 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.498160 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:11.498162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:11.498162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:11.498169 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:11.499696 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:11.500064 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:11.500065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:11.500065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:11.500116 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:11.500362 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:11.500363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:11.500363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:11.500487 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:11.500551 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.500792 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.500861 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:49:11.501116 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.491656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44785 (local address 127.15.232.195:57785)
0810 06:49:11.491852 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:11.491921 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:11.492491 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:11.492755 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:11.492758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:11.493439 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:11.493641 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.493646 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.494386 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.494387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.495200 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.495202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.495783 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.495784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.495935 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.496603 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:11.496614 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:11.498073 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:11.499774 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:11.499781 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:11.499783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:11.499998 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:11.500175 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:11.500176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:11.500176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:11.500277 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:11.500554 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:11.500558 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:11.500675 (+ 117us) client_negotiation.cc:769] Sending connection context
0810 06:49:11.500760 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:49:11.500891 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
I20260810 06:49:11.501642 5067 heartbeater.cc:347] Connected to a master server at 127.15.232.254:44785
I20260810 06:49:11.501734 5067 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:11.502022 5067 heartbeater.cc:511] Master 127.15.232.254:44785 requested a full tablet report, sending...
I20260810 06:49:11.502432 4535 ts_manager.cc:194] Registered new tserver with Master: 7f7b615dd2814c9da40be4052e6ebd6a (127.15.232.195:45173)
I20260810 06:49:11.502835 4535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:57785
I20260810 06:49:11.507675 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:49:11.516515 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.509219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41354 (local address 127.15.232.254:44785)
0810 06:49:11.509275 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.509277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.509393 (+ 116us) server_negotiation.cc:407] Connection header received
0810 06:49:11.509448 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.509449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.509459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.509504 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:11.510069 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.510569 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.511613 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.511868 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.511960 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.513736 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:11.513738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:11.513738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:11.513746 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:11.515385 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:11.515772 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:11.515773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:11.515773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:11.515830 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:11.516039 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:11.516039 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:11.516039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:11.516173 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:11.516256 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.516393 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.516446 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:49:11.518460 4535 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:49:11.520192 4535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:11.527860 5085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.523577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48118 (local address 127.15.232.194:43603)
0810 06:49:11.523861 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.523867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.523878 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:49:11.523929 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.523932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.523949 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.524039 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:11.524259 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.525383 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.526563 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.527424 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.527509 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.527568 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.527715 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.527763 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":88,"threads_started":1}
I20260810 06:49:11.528165 5082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.523256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42040 (local address 127.15.232.193:36901)
0810 06:49:11.523869 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.523874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.523886 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:49:11.523928 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.523930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.523949 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.524072 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:11.524193 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.525041 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.526905 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.527621 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.527722 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.527777 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.528023 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.528077 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":128,"threads_started":1}
I20260810 06:49:11.528234 5080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.523095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:36901 (local address 127.0.0.1:42040)
0810 06:49:11.523304 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:11.523347 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:11.523460 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:11.524073 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:11.524074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:11.524075 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:11.524119 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.524120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.525200 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.525201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.526813 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.526814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.527737 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.527738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.527798 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.527802 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:49:11.527887 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:49:11.527926 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":1,"thread_start_us":87,"threads_started":1}
I20260810 06:49:11.527966 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.523332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:43603 (local address 127.0.0.1:48118)
0810 06:49:11.523657 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:11.523668 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:11.523722 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:11.524131 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:11.524132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:11.524135 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:11.524189 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.524190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.525483 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.525484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.526448 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.526451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.527520 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.527521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.527596 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.527610 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 06:49:11.527680 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:49:11.527867 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":81,"threads_started":1}
I20260810 06:49:11.529232 5083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.523149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:45173 (local address 127.0.0.1:51186)
0810 06:49:11.523593 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:11.523603 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:11.523653 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:11.523983 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:11.523987 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:11.523989 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:11.524058 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.524060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.525460 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.525461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.526811 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.526814 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.528922 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.528923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.528994 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.528999 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:49:11.529067 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:49:11.529111 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":9,"thread_start_us":148,"threads_started":1}
I20260810 06:49:11.529237 5084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.523399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51186 (local address 127.15.232.195:45173)
0810 06:49:11.523566 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.523572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.523715 (+ 143us) server_negotiation.cc:407] Connection header received
0810 06:49:11.523767 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.523770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.523790 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.523868 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:11.524147 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.525332 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.527819 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.528807 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.528905 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.528961 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.529089 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.529130 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260810 06:49:11.529383 4834 tablet_service.cc:1467] Processing CreateTablet for tablet 68985cbca24f466bbdfb3590fe10aa39 (DEFAULT_TABLE table=test-table [id=2b262a33fb304f0eac7449e5304fbb78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:11.529399 4683 tablet_service.cc:1467] Processing CreateTablet for tablet 68985cbca24f466bbdfb3590fe10aa39 (DEFAULT_TABLE table=test-table [id=2b262a33fb304f0eac7449e5304fbb78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:11.529779 4834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68985cbca24f466bbdfb3590fe10aa39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:11.529788 4683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68985cbca24f466bbdfb3590fe10aa39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:11.530220 4991 tablet_service.cc:1467] Processing CreateTablet for tablet 68985cbca24f466bbdfb3590fe10aa39 (DEFAULT_TABLE table=test-table [id=2b262a33fb304f0eac7449e5304fbb78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:11.530546 4991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68985cbca24f466bbdfb3590fe10aa39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:11.532788 5086 tablet_bootstrap.cc:492] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4: Bootstrap starting.
I20260810 06:49:11.533672 5086 tablet_bootstrap.cc:654] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:11.533874 5087 tablet_bootstrap.cc:492] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372: Bootstrap starting.
I20260810 06:49:11.534018 5088 tablet_bootstrap.cc:492] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a: Bootstrap starting.
I20260810 06:49:11.534363 5086 log.cc:824] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:11.534672 5088 tablet_bootstrap.cc:654] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:11.534704 5087 tablet_bootstrap.cc:654] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:11.535272 5088 log.cc:824] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:11.535321 5086 tablet_bootstrap.cc:492] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4: No bootstrap required, opened a new log
I20260810 06:49:11.535396 5086 ts_tablet_manager.cc:1397] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:49:11.535569 5087 log.cc:824] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:11.536127 5088 tablet_bootstrap.cc:492] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a: No bootstrap required, opened a new log
I20260810 06:49:11.536197 5088 ts_tablet_manager.cc:1397] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:49:11.537595 5087 tablet_bootstrap.cc:492] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372: No bootstrap required, opened a new log
I20260810 06:49:11.537673 5087 ts_tablet_manager.cc:1397] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 06:49:11.537669 5086 raft_consensus.cc:348] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b0fc91c5246138a920496e61994d4" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36901 } } peers { permanent_uuid: "8685d11d976946e89a4781928ec66372" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 43603 } } peers { permanent_uuid: "7f7b615dd2814c9da40be4052e6ebd6a" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45173 } }
I20260810 06:49:11.537889 5086 raft_consensus.cc:374] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:11.537932 5086 raft_consensus.cc:729] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270b0fc91c5246138a920496e61994d4, State: Initialized, Role: FOLLOWER
I20260810 06:49:11.538053 5086 consensus_queue.cc:260] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b0fc91c5246138a920496e61994d4" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36901 } } peers { permanent_uuid: "8685d11d976946e89a4781928ec66372" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 43603 } } peers { permanent_uuid: "7f7b615dd2814c9da40be4052e6ebd6a" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45173 } }
I20260810 06:49:11.538347 5086 ts_tablet_manager.cc:1428] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:49:11.538347 4756 heartbeater.cc:503] Master 127.15.232.254:44785 was elected leader, sending a full tablet report...
I20260810 06:49:11.538328 5088 raft_consensus.cc:348] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b0fc91c5246138a920496e61994d4" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36901 } } peers { permanent_uuid: "8685d11d976946e89a4781928ec66372" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 43603 } } peers { permanent_uuid: "7f7b615dd2814c9da40be4052e6ebd6a" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45173 } }
I20260810 06:49:11.538488 5088 raft_consensus.cc:374] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:11.538524 5088 raft_consensus.cc:729] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f7b615dd2814c9da40be4052e6ebd6a, State: Initialized, Role: FOLLOWER
I20260810 06:49:11.538658 5088 consensus_queue.cc:260] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b0fc91c5246138a920496e61994d4" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36901 } } peers { permanent_uuid: "8685d11d976946e89a4781928ec66372" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 43603 } } peers { permanent_uuid: "7f7b615dd2814c9da40be4052e6ebd6a" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45173 } }
I20260810 06:49:11.538937 5088 ts_tablet_manager.cc:1428] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:49:11.539069 5067 heartbeater.cc:503] Master 127.15.232.254:44785 was elected leader, sending a full tablet report...
I20260810 06:49:11.539676 5087 raft_consensus.cc:348] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b0fc91c5246138a920496e61994d4" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36901 } } peers { permanent_uuid: "8685d11d976946e89a4781928ec66372" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 43603 } } peers { permanent_uuid: "7f7b615dd2814c9da40be4052e6ebd6a" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45173 } }
I20260810 06:49:11.539803 5087 raft_consensus.cc:374] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:11.539870 5087 raft_consensus.cc:729] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8685d11d976946e89a4781928ec66372, State: Initialized, Role: FOLLOWER
I20260810 06:49:11.539991 5087 consensus_queue.cc:260] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b0fc91c5246138a920496e61994d4" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36901 } } peers { permanent_uuid: "8685d11d976946e89a4781928ec66372" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 43603 } } peers { permanent_uuid: "7f7b615dd2814c9da40be4052e6ebd6a" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45173 } }
I20260810 06:49:11.540273 5087 ts_tablet_manager.cc:1428] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:49:11.540287 4912 heartbeater.cc:503] Master 127.15.232.254:44785 was elected leader, sending a full tablet report...
W20260810 06:49:11.564507 4913 tablet.cc:2369] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:49:11.565394 5096 raft_consensus.cc:482] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:11.565536 5096 raft_consensus.cc:504] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b0fc91c5246138a920496e61994d4" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36901 } } peers { permanent_uuid: "8685d11d976946e89a4781928ec66372" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 43603 } } peers { permanent_uuid: "7f7b615dd2814c9da40be4052e6ebd6a" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45173 } }
I20260810 06:49:11.566033 5096 leader_election.cc:288] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 270b0fc91c5246138a920496e61994d4 (127.15.232.193:36901), 7f7b615dd2814c9da40be4052e6ebd6a (127.15.232.195:45173)
I20260810 06:49:11.569394 4914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.566240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:36901 (local address 127.15.232.194:53971)
0810 06:49:11.566347 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:11.566362 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:11.566450 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:11.566691 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:11.566692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:11.566694 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:11.566780 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.566782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.567555 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.567556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.568351 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.568353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.569084 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.569085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.569201 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.569205 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:49:11.569262 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 06:49:11.569296 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":1}
I20260810 06:49:11.569415 5082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.566322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:53971 (local address 127.15.232.193:36901)
0810 06:49:11.566387 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.566390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.566503 (+ 113us) server_negotiation.cc:407] Connection header received
0810 06:49:11.566548 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.566549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.566564 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.566672 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:11.566869 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.567460 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.568465 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.568989 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.569055 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.569064 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.569266 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.569320 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:49:11.569505 5084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.566353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:34763 (local address 127.15.232.195:45173)
0810 06:49:11.566446 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.566449 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.566682 (+ 233us) server_negotiation.cc:407] Connection header received
0810 06:49:11.566758 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.566759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.566772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.566861 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:11.567062 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.567570 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.568615 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.569016 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.569119 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.569127 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.569381 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.569433 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:49:11.569676 5097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.566243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:45173 (local address 127.15.232.194:34763)
0810 06:49:11.566559 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:11.566569 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:11.566652 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:11.566915 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:11.566915 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:11.566917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:11.566965 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.566966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.567654 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.567655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.568518 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.568521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.569143 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.569144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.569287 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.569292 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:49:11.569354 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 06:49:11.569397 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":74,"thread_start_us":166,"threads_started":1}
I20260810 06:49:11.569868 4705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68985cbca24f466bbdfb3590fe10aa39" candidate_uuid: "8685d11d976946e89a4781928ec66372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270b0fc91c5246138a920496e61994d4" is_pre_election: true
I20260810 06:49:11.570071 4705 raft_consensus.cc:2454] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8685d11d976946e89a4781928ec66372 in term 0.
I20260810 06:49:11.570041 5015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68985cbca24f466bbdfb3590fe10aa39" candidate_uuid: "8685d11d976946e89a4781928ec66372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f7b615dd2814c9da40be4052e6ebd6a" is_pre_election: true
I20260810 06:49:11.570171 5015 raft_consensus.cc:2454] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8685d11d976946e89a4781928ec66372 in term 0.
I20260810 06:49:11.570330 4784 leader_election.cc:302] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 270b0fc91c5246138a920496e61994d4, 8685d11d976946e89a4781928ec66372; no voters:
I20260810 06:49:11.570490 5096 raft_consensus.cc:2781] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:11.570586 5096 raft_consensus.cc:482] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:11.570626 5096 raft_consensus.cc:3037] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:11.571506 5096 raft_consensus.cc:504] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b0fc91c5246138a920496e61994d4" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36901 } } peers { permanent_uuid: "8685d11d976946e89a4781928ec66372" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 43603 } } peers { permanent_uuid: "7f7b615dd2814c9da40be4052e6ebd6a" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45173 } }
I20260810 06:49:11.571686 5096 leader_election.cc:288] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [CANDIDATE]: Term 1 election: Requested vote from peers 270b0fc91c5246138a920496e61994d4 (127.15.232.193:36901), 7f7b615dd2814c9da40be4052e6ebd6a (127.15.232.195:45173)
I20260810 06:49:11.571830 5015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68985cbca24f466bbdfb3590fe10aa39" candidate_uuid: "8685d11d976946e89a4781928ec66372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f7b615dd2814c9da40be4052e6ebd6a"
I20260810 06:49:11.571853 4705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68985cbca24f466bbdfb3590fe10aa39" candidate_uuid: "8685d11d976946e89a4781928ec66372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270b0fc91c5246138a920496e61994d4"
I20260810 06:49:11.571923 5015 raft_consensus.cc:3037] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:11.571924 4705 raft_consensus.cc:3037] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:11.572890 5015 raft_consensus.cc:2454] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8685d11d976946e89a4781928ec66372 in term 1.
I20260810 06:49:11.572886 4705 raft_consensus.cc:2454] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8685d11d976946e89a4781928ec66372 in term 1.
I20260810 06:49:11.573118 4783 leader_election.cc:302] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f7b615dd2814c9da40be4052e6ebd6a, 8685d11d976946e89a4781928ec66372; no voters:
I20260810 06:49:11.573268 5096 raft_consensus.cc:2781] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:11.573500 5096 raft_consensus.cc:686] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 1 LEADER]: Becoming Leader. State: Replica: 8685d11d976946e89a4781928ec66372, State: Running, Role: LEADER
I20260810 06:49:11.573632 5096 consensus_queue.cc:237] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b0fc91c5246138a920496e61994d4" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36901 } } peers { permanent_uuid: "8685d11d976946e89a4781928ec66372" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 43603 } } peers { permanent_uuid: "7f7b615dd2814c9da40be4052e6ebd6a" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45173 } }
I20260810 06:49:11.574543 4535 catalog_manager.cc:5495] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8685d11d976946e89a4781928ec66372 (127.15.232.194). New cstate: current_term: 1 leader_uuid: "8685d11d976946e89a4781928ec66372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b0fc91c5246138a920496e61994d4" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8685d11d976946e89a4781928ec66372" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f7b615dd2814c9da40be4052e6ebd6a" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45173 } health_report { overall_health: UNKNOWN } } }
I20260810 06:49:11.589440 5085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.586791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48120 (local address 127.15.232.194:43603)
0810 06:49:11.586904 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.586906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.586917 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:49:11.586957 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.586958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.586975 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.587045 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:11.587227 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.587698 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.588484 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.588696 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.588766 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.588960 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:11.589056 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:11.589139 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.589340 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.589351 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 06:49:11.591466 4705 raft_consensus.cc:1261] T 68985cbca24f466bbdfb3590fe10aa39 P 270b0fc91c5246138a920496e61994d4 [term 1 FOLLOWER]: Refusing update from remote peer 8685d11d976946e89a4781928ec66372: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:11.591466 5015 raft_consensus.cc:1261] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a [term 1 FOLLOWER]: Refusing update from remote peer 8685d11d976946e89a4781928ec66372: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:11.591995 5098 consensus_queue.cc:1035] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "270b0fc91c5246138a920496e61994d4" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:11.592140 5096 consensus_queue.cc:1035] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f7b615dd2814c9da40be4052e6ebd6a" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:11.594995 5102 mvcc.cc:205] Tried to move back new op lower bound from 7316867283315621888 to 7316867283246186496. Current Snapshot: MvccSnapshot[applied={T|T < 7316867283315621888}]
I20260810 06:49:11.595723 5103 mvcc.cc:205] Tried to move back new op lower bound from 7316867283315621888 to 7316867283246186496. Current Snapshot: MvccSnapshot[applied={T|T < 7316867283315621888}]
I20260810 06:49:11.595899 5104 mvcc.cc:205] Tried to move back new op lower bound from 7316867283315621888 to 7316867283246186496. Current Snapshot: MvccSnapshot[applied={T|T < 7316867283315621888 or (T in {7316867283315621888})}]
I20260810 06:49:11.597645 4533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:11.597755 4533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:11.598587 4533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3941c0ef87145f4bd7054057a4964fe: Sending DeleteTablet for 3 replicas of tablet 68985cbca24f466bbdfb3590fe10aa39
I20260810 06:49:11.598991 4991 tablet_service.cc:1514] Processing DeleteTablet for tablet 68985cbca24f466bbdfb3590fe10aa39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:11 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:51186
I20260810 06:49:11.599211 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 4607
I20260810 06:49:11.601792 4834 tablet_service.cc:1514] Processing DeleteTablet for tablet 68985cbca24f466bbdfb3590fe10aa39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:11 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:48118
I20260810 06:49:11.602435 5116 tablet_replica.cc:330] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372: stopping tablet replica
I20260810 06:49:11.602545 5116 raft_consensus.cc:2229] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:49:11.602689 5116 raft_consensus.cc:2258] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:11.603514 5116 ts_tablet_manager.cc:1905] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:11.605748 5116 ts_tablet_manager.cc:1918] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:11.605837 5116 log.cc:1196] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/68985cbca24f466bbdfb3590fe10aa39
I20260810 06:49:11.606175 5116 ts_tablet_manager.cc:1939] T 68985cbca24f466bbdfb3590fe10aa39 P 8685d11d976946e89a4781928ec66372: Deleting consensus metadata
I20260810 06:49:11.606704 4520 catalog_manager.cc:4841] TS 8685d11d976946e89a4781928ec66372 (127.15.232.194:43603): tablet 68985cbca24f466bbdfb3590fe10aa39 (table test-table [id=2b262a33fb304f0eac7449e5304fbb78]) successfully deleted
I20260810 06:49:11.614157 5115 tablet_replica.cc:330] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a: stopping tablet replica
I20260810 06:49:11.614384 5115 raft_consensus.cc:2229] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:11.614539 5115 raft_consensus.cc:2258] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 06:49:11.614766 4520 connection.cc:535] client connection to 127.15.232.193:36901 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 06:49:11.614897 4520 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:49:11.615235 5115 ts_tablet_manager.cc:1905] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:11.615446 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.615320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:36901 (local address 127.0.0.1:42042)
0810 06:49:11.615375 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:11.615410 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:36901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 06:49:11.615578 4520 catalog_manager.cc:4584] TS 270b0fc91c5246138a920496e61994d4 (127.15.232.193:36901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 68985cbca24f466bbdfb3590fe10aa39: Network error: Client connection negotiation failed: client connection to 127.15.232.193:36901: connect: Connection refused (error 111)
I20260810 06:49:11.615996 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 4761
I20260810 06:49:11.617588 5115 ts_tablet_manager.cc:1918] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:11.617681 5115 log.cc:1196] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/68985cbca24f466bbdfb3590fe10aa39
I20260810 06:49:11.617972 5115 ts_tablet_manager.cc:1939] T 68985cbca24f466bbdfb3590fe10aa39 P 7f7b615dd2814c9da40be4052e6ebd6a: Deleting consensus metadata
I20260810 06:49:11.618511 4519 catalog_manager.cc:4841] TS 7f7b615dd2814c9da40be4052e6ebd6a (127.15.232.195:45173): tablet 68985cbca24f466bbdfb3590fe10aa39 (table test-table [id=2b262a33fb304f0eac7449e5304fbb78]) successfully deleted
I20260810 06:49:11.623709 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 4918
I20260810 06:49:11.632277 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 4501
2026-08-10T06:49:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2082 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T06:49:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:11Z Disabled control of system clock
I20260810 06:49:11.664137 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:38553
--webserver_interface=127.15.232.254
--webserver_port=0
--builtin_ntp_servers=127.15.232.212:44503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:38553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:49:11.748744 5121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:11.748970 5121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:11.748991 5121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:11.750196 5121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:11.750241 5121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:11.750254 5121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:11.750267 5121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:11.750278 5121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:11.751701 5121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:38553
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:38553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:11.751967 5121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:11.752287 5121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:11.757388 5129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:11.757405 5128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:11.757407 5121 server_base.cc:1034] running on GCE node
W20260810 06:49:11.757390 5131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:11.757982 5121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:11.758246 5121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:11.759416 5121 hybrid_clock.cc:648] HybridClock initialized: now 1786344551759384 us; error 42 us; skew 500 ppm
I20260810 06:49:11.760776 5121 webserver.cc:460] Webserver started at http://127.15.232.254:40801/ using document root <none> and password file <none>
I20260810 06:49:11.761019 5121 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:11.761067 5121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:11.761238 5121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:11.762418 5121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "1fc542a1bd9143fbae7754f483d9d6b0"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "40677568028ed2e23dc935dfdb4ff514"
server_key_iv: "387dc499a97581f04a649eda4a1ddccc"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.762732 5121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "1fc542a1bd9143fbae7754f483d9d6b0"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "40677568028ed2e23dc935dfdb4ff514"
server_key_iv: "387dc499a97581f04a649eda4a1ddccc"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.764096 5121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:49:11.764945 5138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.765214 5121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:49:11.765321 5121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "1fc542a1bd9143fbae7754f483d9d6b0"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "40677568028ed2e23dc935dfdb4ff514"
server_key_iv: "387dc499a97581f04a649eda4a1ddccc"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.765389 5121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:11.775182 5121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:11.775491 5121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:11.775604 5121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:11.779647 5121 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:38553
I20260810 06:49:11.779747 5198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:38553 every 8 connection(s)
I20260810 06:49:11.780094 5121 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:11.781351 5199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:11.783901 5199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0: Bootstrap starting.
I20260810 06:49:11.784569 5199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:11.784905 5199 log.cc:824] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:11.785892 5199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0: No bootstrap required, opened a new log
I20260810 06:49:11.787611 5199 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc542a1bd9143fbae7754f483d9d6b0" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38553 } }
I20260810 06:49:11.787806 5199 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:11.787889 5199 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc542a1bd9143fbae7754f483d9d6b0, State: Initialized, Role: FOLLOWER
I20260810 06:49:11.788072 5199 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc542a1bd9143fbae7754f483d9d6b0" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38553 } }
I20260810 06:49:11.788160 5199 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:11.788219 5199 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:11.788269 5199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:11.788841 5199 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc542a1bd9143fbae7754f483d9d6b0" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38553 } }
I20260810 06:49:11.788966 5199 leader_election.cc:302] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fc542a1bd9143fbae7754f483d9d6b0; no voters:
I20260810 06:49:11.789211 5199 leader_election.cc:288] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:11.789288 5203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:11.789430 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 5121
I20260810 06:49:11.789497 5199 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:11.789517 5203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [term 1 LEADER]: Becoming Leader. State: Replica: 1fc542a1bd9143fbae7754f483d9d6b0, State: Running, Role: LEADER
I20260810 06:49:11.789565 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:11.789633 5203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc542a1bd9143fbae7754f483d9d6b0" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38553 } }
I20260810 06:49:11.789806 16291 external_mini_cluster.cc:1428] Setting key 6a4d5f4228a4f8c817e31ff5f165df3e
I20260810 06:49:11.790055 5203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fc542a1bd9143fbae7754f483d9d6b0. Latest consensus state: current_term: 1 leader_uuid: "1fc542a1bd9143fbae7754f483d9d6b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc542a1bd9143fbae7754f483d9d6b0" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38553 } } }
I20260810 06:49:11.790136 5203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:11.790725 5204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fc542a1bd9143fbae7754f483d9d6b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc542a1bd9143fbae7754f483d9d6b0" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38553 } } }
I20260810 06:49:11.790884 5204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:11.790938 5219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:11.791476 5219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:11.793273 5219 catalog_manager.cc:1347] Generated new cluster ID: b4d7ab9abda14cac9285287b10f36dcb
I20260810 06:49:11.793332 5219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:11.796408 5221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.790421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58774 (local address 127.15.232.254:38553)
0810 06:49:11.791001 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.791011 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.791044 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:49:11.791168 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.791175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.791260 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.791459 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:11.791734 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.792954 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.794130 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.794723 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.794851 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.795644 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:11.795647 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:11.795648 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:11.795886 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:11.795927 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:11.795941 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:11.796004 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.796109 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.796188 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":83,"threads_started":1}
I20260810 06:49:11.806854 5219 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:11.807634 5219 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:11.816471 5219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0: Generated new TSK 0
I20260810 06:49:11.816774 5219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:11.824355 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:38553
--builtin_ntp_servers=127.15.232.212:44503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:49:11.909119 5234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:11.909420 5234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:11.909462 5234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:11.910702 5234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:11.910791 5234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:11.910836 5234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:49:11.912256 5234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:38553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:11.912565 5234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:11.912904 5234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:11.918339 5244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:11.918421 5240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:11.918349 5241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:11.918684 5234 server_base.cc:1034] running on GCE node
I20260810 06:49:11.918908 5234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:11.919189 5234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:11.920373 5234 hybrid_clock.cc:648] HybridClock initialized: now 1786344551920356 us; error 31 us; skew 500 ppm
I20260810 06:49:11.921856 5234 webserver.cc:460] Webserver started at http://127.15.232.193:33223/ using document root <none> and password file <none>
I20260810 06:49:11.922166 5234 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:11.922247 5234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:11.922410 5234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:11.923552 5234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "73fbaaf55700488685538c6f57ddceba"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "be1c8fc19379db151cb458fdab5db840"
server_key_iv: "52182f0563442c8a59ed640a31def6ce"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.923950 5234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "73fbaaf55700488685538c6f57ddceba"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "be1c8fc19379db151cb458fdab5db840"
server_key_iv: "52182f0563442c8a59ed640a31def6ce"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.925526 5234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260810 06:49:11.926481 5252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.926680 5234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:11.926779 5234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "73fbaaf55700488685538c6f57ddceba"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-vxj2"
server_key: "be1c8fc19379db151cb458fdab5db840"
server_key_iv: "52182f0563442c8a59ed640a31def6ce"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.926913 5234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:11.945832 5234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:11.946187 5234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:11.946348 5234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:11.946619 5234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:11.946991 5234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:11.947047 5234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.947088 5234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:11.947109 5234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.953372 5234 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:36203
I20260810 06:49:11.953435 5390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:36203 every 8 connection(s)
I20260810 06:49:11.953773 5234 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:49:11.959412 5221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.954546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:48423 (local address 127.15.232.254:38553)
0810 06:49:11.954597 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.954599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.954900 (+ 301us) server_negotiation.cc:407] Connection header received
0810 06:49:11.955435 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.955436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.955450 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.955497 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:11.956174 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.956683 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.957746 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.957960 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.958023 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.959116 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:11.959118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:11.959118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:11.959123 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:11.959138 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:11.959142 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:11.959190 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.959301 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.959348 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:49:11.959612 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.954504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:38553 (local address 127.15.232.193:48423)
0810 06:49:11.954752 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:11.954806 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:11.955315 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:11.955573 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:11.955577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:11.955585 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:11.956094 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.956099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.956770 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.956771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.957659 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:11.957662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.958086 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.958088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:11.958218 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.958954 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:49:11.958954 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:11.958986 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:49:11.959014 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:11.959205 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:11.959210 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:11.959212 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:49:11.959292 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:49:11.959393 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":141,"threads_started":1}
I20260810 06:49:11.960283 5391 heartbeater.cc:347] Connected to a master server at 127.15.232.254:38553
I20260810 06:49:11.960405 5391 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:11.960487 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 5234
I20260810 06:49:11.960603 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:49:11.960636 5391 heartbeater.cc:511] Master 127.15.232.254:38553 requested a full tablet report, sending...
I20260810 06:49:11.960814 16291 external_mini_cluster.cc:1428] Setting key 9436a5ebb953f13f369e72d78177926a
I20260810 06:49:11.961340 5158 ts_manager.cc:194] Registered new tserver with Master: 73fbaaf55700488685538c6f57ddceba (127.15.232.193:36203)
I20260810 06:49:11.962008 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:38553
--builtin_ntp_servers=127.15.232.212:44503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:49:11.962045 5158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.193:48423
W20260810 06:49:12.049050 5394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:12.049369 5394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:12.049396 5394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:12.050668 5394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:12.050715 5394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:12.050770 5394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:49:12.052224 5394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:38553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:12.052505 5394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:12.052848 5394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:12.058602 5400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:12.058602 5402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:12.058606 5404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:12.058923 5394 server_base.cc:1034] running on GCE node
I20260810 06:49:12.059134 5394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:12.059366 5394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:12.060519 5394 hybrid_clock.cc:648] HybridClock initialized: now 1786344552060497 us; error 37 us; skew 500 ppm
I20260810 06:49:12.062062 5394 webserver.cc:460] Webserver started at http://127.15.232.194:35581/ using document root <none> and password file <none>
I20260810 06:49:12.062352 5394 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:12.062405 5394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:12.062589 5394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:12.063784 5394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "186a313b0be84814b8ee6498852e15ff"
format_stamp: "Formatted at 2026-08-10 06:49:12 on dist-test-slave-vxj2"
server_key: "00080893001585cbda632f5b658ad9b3"
server_key_iv: "2624dc74ccca00d2cc72b080c21d4581"
server_key_version: "encryptionkey@0"
I20260810 06:49:12.064172 5394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "186a313b0be84814b8ee6498852e15ff"
format_stamp: "Formatted at 2026-08-10 06:49:12 on dist-test-slave-vxj2"
server_key: "00080893001585cbda632f5b658ad9b3"
server_key_iv: "2624dc74ccca00d2cc72b080c21d4581"
server_key_version: "encryptionkey@0"
I20260810 06:49:12.065654 5394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:49:12.066447 5410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:12.066634 5394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:12.066731 5394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "186a313b0be84814b8ee6498852e15ff"
format_stamp: "Formatted at 2026-08-10 06:49:12 on dist-test-slave-vxj2"
server_key: "00080893001585cbda632f5b658ad9b3"
server_key_iv: "2624dc74ccca00d2cc72b080c21d4581"
server_key_version: "encryptionkey@0"
I20260810 06:49:12.066809 5394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:12.090592 5394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:12.091009 5394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:12.091189 5394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:12.091477 5394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:12.091853 5394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:12.091909 5394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:12.091955 5394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:12.091984 5394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:12.098228 5394 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:36355
I20260810 06:49:12.098287 5550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:36355 every 8 connection(s)
I20260810 06:49:12.098686 5394 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:49:12.104168 5221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.099542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:53537 (local address 127.15.232.254:38553)
0810 06:49:12.099602 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:12.099604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:12.099854 (+ 250us) server_negotiation.cc:407] Connection header received
0810 06:49:12.100341 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:12.100341 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:12.100351 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:12.100418 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:12.101271 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.101770 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.102756 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.102960 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.103018 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.103878 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:12.103880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:12.103880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:12.103885 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:12.103900 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:12.103904 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:12.103966 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:12.104076 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 06:49:12.104102 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:49:12.104312 5555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.099507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:38553 (local address 127.15.232.194:53537)
0810 06:49:12.099705 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:12.099748 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:12.100254 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:12.100487 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:12.100491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:12.100500 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:12.101093 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.101098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.101859 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.101860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.102667 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.102669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.103043 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.103043 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.103162 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.103718 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:49:12.103718 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:12.103747 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:49:12.103771 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:12.103973 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:12.103977 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:12.103980 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:49:12.104034 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 06:49:12.104118 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260810 06:49:12.104857 5551 heartbeater.cc:347] Connected to a master server at 127.15.232.254:38553
I20260810 06:49:12.104987 5551 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:12.105217 5551 heartbeater.cc:511] Master 127.15.232.254:38553 requested a full tablet report, sending...
I20260810 06:49:12.105619 5158 ts_manager.cc:194] Registered new tserver with Master: 186a313b0be84814b8ee6498852e15ff (127.15.232.194:36355)
I20260810 06:49:12.106045 5158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.194:53537
I20260810 06:49:12.108385 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 5394
I20260810 06:49:12.108485 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:49:12.108768 16291 external_mini_cluster.cc:1428] Setting key 2a2222b92a3fafe1f04905714fa0f399
I20260810 06:49:12.109879 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:38553
--builtin_ntp_servers=127.15.232.212:44503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:49:12.196571 5556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:12.196833 5556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:12.196859 5556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:12.198134 5556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:12.198184 5556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:12.198211 5556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:49:12.199630 5556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:38553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:12.199940 5556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:12.200291 5556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:12.206269 5563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:12.206439 5556 server_base.cc:1034] running on GCE node
W20260810 06:49:12.206302 5564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:12.206269 5566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:12.206789 5556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:12.207083 5556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:12.208235 5556 hybrid_clock.cc:648] HybridClock initialized: now 1786344552208215 us; error 45 us; skew 500 ppm
I20260810 06:49:12.209738 5556 webserver.cc:460] Webserver started at http://127.15.232.195:44849/ using document root <none> and password file <none>
I20260810 06:49:12.210068 5556 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:12.210148 5556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:12.210318 5556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:12.211498 5556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e"
format_stamp: "Formatted at 2026-08-10 06:49:12 on dist-test-slave-vxj2"
server_key: "acd17bbb32f34af46ba8ba30914467f8"
server_key_iv: "dbbfb96d1f6392119bebc016773c73cc"
server_key_version: "encryptionkey@0"
I20260810 06:49:12.211905 5556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e"
format_stamp: "Formatted at 2026-08-10 06:49:12 on dist-test-slave-vxj2"
server_key: "acd17bbb32f34af46ba8ba30914467f8"
server_key_iv: "dbbfb96d1f6392119bebc016773c73cc"
server_key_version: "encryptionkey@0"
I20260810 06:49:12.213594 5556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260810 06:49:12.214663 5573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:12.214905 5556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:12.215026 5556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e"
format_stamp: "Formatted at 2026-08-10 06:49:12 on dist-test-slave-vxj2"
server_key: "acd17bbb32f34af46ba8ba30914467f8"
server_key_iv: "dbbfb96d1f6392119bebc016773c73cc"
server_key_version: "encryptionkey@0"
I20260810 06:49:12.215124 5556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:12.228323 5556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:12.228632 5556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:12.228742 5556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:12.228950 5556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:12.229411 5556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:12.229455 5556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:12.229480 5556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:12.229529 5556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:12.235966 5556 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:38757
I20260810 06:49:12.236052 5716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:38757 every 8 connection(s)
I20260810 06:49:12.236389 5556 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:49:12.242069 5221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.237311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:47717 (local address 127.15.232.254:38553)
0810 06:49:12.237359 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:12.237361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:12.237637 (+ 276us) server_negotiation.cc:407] Connection header received
0810 06:49:12.238193 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:12.238195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:12.238207 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:12.238270 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:12.238946 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.239458 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.240478 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.240676 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.240735 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.241742 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:12.241744 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:12.241744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:12.241749 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:12.241765 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:12.241769 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:12.241827 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:12.241967 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 06:49:12.242000 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:49:12.242286 5719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.237324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:38553 (local address 127.15.232.195:47717)
0810 06:49:12.237508 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:12.237543 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:12.238072 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:12.238324 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:12.238328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:12.238336 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:12.238853 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.238858 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.239547 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.239548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.240392 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.240395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.240758 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.240758 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.240904 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.241563 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:49:12.241564 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:12.241595 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:49:12.241621 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:12.241840 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:12.241844 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:12.241847 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:49:12.241935 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 06:49:12.242031 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260810 06:49:12.243011 5717 heartbeater.cc:347] Connected to a master server at 127.15.232.254:38553
I20260810 06:49:12.243130 5717 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:12.243296 5717 heartbeater.cc:511] Master 127.15.232.254:38553 requested a full tablet report, sending...
I20260810 06:49:12.243736 5158 ts_manager.cc:194] Registered new tserver with Master: 2ff478ad7b374d8095ee9e04d6ca6c2e (127.15.232.195:38757)
I20260810 06:49:12.244151 5158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.195:47717
I20260810 06:49:12.245719 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 5556
I20260810 06:49:12.245817 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:49:12.246047 16291 external_mini_cluster.cc:1428] Setting key 86fb519118d960de4182901abb6e4dd2
I20260810 06:49:12.247102 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:49:12.252250 5221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.248598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58786 (local address 127.15.232.254:38553)
0810 06:49:12.248662 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:12.248665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:12.248824 (+ 159us) server_negotiation.cc:407] Connection header received
0810 06:49:12.248877 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:12.248879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:12.248894 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:12.248952 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:12.249143 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.249734 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.250650 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.250906 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.251008 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.251872 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:12.251873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:12.251873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:12.251878 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:12.251896 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:12.251900 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:12.252028 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:12.252135 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 06:49:12.252148 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:49:12.254369 5158 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:49:12.256403 5158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:12.263689 5729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.259275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45960 (local address 127.15.232.193:36203)
0810 06:49:12.259698 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:12.259704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:12.259714 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:49:12.259753 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:12.259754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:12.259768 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:12.259837 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:12.260050 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.260830 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.262405 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.263217 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.263296 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.263396 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:12.263534 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 06:49:12.263586 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":102,"threads_started":1}
I20260810 06:49:12.263687 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.259079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:36203 (local address 127.0.0.1:45960)
0810 06:49:12.259297 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:12.259346 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:12.259421 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:12.259851 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:12.259851 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:12.259854 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:12.259937 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.259939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.260927 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.260929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.262255 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.262258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.263301 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.263302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.263392 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.263408 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 06:49:12.263498 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 06:49:12.263555 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":73,"threads_started":1}
I20260810 06:49:12.263705 5725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.259080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:36355 (local address 127.0.0.1:44362)
0810 06:49:12.259486 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:12.259496 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:12.259550 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:12.259746 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:12.259749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:12.259752 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:12.260043 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.260046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.261396 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.261397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.262428 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.262430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.263432 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.263433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.263509 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.263514 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:49:12.263570 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 06:49:12.263611 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":8,"spinlock_wait_cycles":896,"thread_start_us":144,"threads_started":1}
I20260810 06:49:12.264027 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.259212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44362 (local address 127.15.232.194:36355)
0810 06:49:12.259454 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:12.259460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:12.259553 (+ 93us) server_negotiation.cc:407] Connection header received
0810 06:49:12.259629 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:12.259632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:12.259651 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:12.259730 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:12.260134 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.261311 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.262508 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.263213 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.263297 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.263396 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:12.263742 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 06:49:12.263793 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
I20260810 06:49:12.264725 5313 tablet_service.cc:1467] Processing CreateTablet for tablet cfbf363d805b4c0f863bf14f7328b192 (DEFAULT_TABLE table=test-table [id=e25b2fbe53db49c9bf881719661982cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:12.264899 5471 tablet_service.cc:1467] Processing CreateTablet for tablet cfbf363d805b4c0f863bf14f7328b192 (DEFAULT_TABLE table=test-table [id=e25b2fbe53db49c9bf881719661982cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:12.265101 5730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.259335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38757 (local address 127.0.0.1:37478)
0810 06:49:12.259568 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:12.259579 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:12.259637 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:12.259987 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:12.259988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:12.259990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:12.260044 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.260046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.262371 (+ 2325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.262372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.263114 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.263116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.264826 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.264828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.264895 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.264900 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:49:12.264967 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 06:49:12.265016 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":73,"threads_started":1}
I20260810 06:49:12.265203 5313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfbf363d805b4c0f863bf14f7328b192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:12.265352 5471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfbf363d805b4c0f863bf14f7328b192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:12.265439 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.259538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37478 (local address 127.15.232.195:38757)
0810 06:49:12.259745 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:12.259749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:12.259758 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:49:12.259797 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:12.259800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:12.259814 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:12.259915 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:12.260133 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.261298 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.263216 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.264617 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.264694 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.264785 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:12.265297 (+ 512us) server_negotiation.cc:299] Negotiation successful
0810 06:49:12.265351 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":94,"threads_started":1}
I20260810 06:49:12.266077 5633 tablet_service.cc:1467] Processing CreateTablet for tablet cfbf363d805b4c0f863bf14f7328b192 (DEFAULT_TABLE table=test-table [id=e25b2fbe53db49c9bf881719661982cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:12.266420 5633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfbf363d805b4c0f863bf14f7328b192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:12.268252 5732 tablet_bootstrap.cc:492] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff: Bootstrap starting.
I20260810 06:49:12.269001 5733 tablet_bootstrap.cc:492] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba: Bootstrap starting.
I20260810 06:49:12.269040 5732 tablet_bootstrap.cc:654] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:12.269589 5732 log.cc:824] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:12.269765 5733 tablet_bootstrap.cc:654] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:12.270062 5736 tablet_bootstrap.cc:492] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e: Bootstrap starting.
I20260810 06:49:12.270120 5733 log.cc:824] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:12.270481 5732 tablet_bootstrap.cc:492] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff: No bootstrap required, opened a new log
I20260810 06:49:12.270574 5732 ts_tablet_manager.cc:1397] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:49:12.271023 5736 tablet_bootstrap.cc:654] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:12.271196 5733 tablet_bootstrap.cc:492] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba: No bootstrap required, opened a new log
I20260810 06:49:12.271335 5733 ts_tablet_manager.cc:1397] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:49:12.271454 5736 log.cc:824] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:12.272128 5736 tablet_bootstrap.cc:492] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e: No bootstrap required, opened a new log
I20260810 06:49:12.272184 5736 ts_tablet_manager.cc:1397] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:49:12.273594 5732 raft_consensus.cc:348] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73fbaaf55700488685538c6f57ddceba" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36203 } } peers { permanent_uuid: "186a313b0be84814b8ee6498852e15ff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36355 } } peers { permanent_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38757 } }
I20260810 06:49:12.273787 5732 raft_consensus.cc:374] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:12.273825 5732 raft_consensus.cc:729] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186a313b0be84814b8ee6498852e15ff, State: Initialized, Role: FOLLOWER
I20260810 06:49:12.273980 5732 consensus_queue.cc:260] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73fbaaf55700488685538c6f57ddceba" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36203 } } peers { permanent_uuid: "186a313b0be84814b8ee6498852e15ff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36355 } } peers { permanent_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38757 } }
I20260810 06:49:12.274289 5732 ts_tablet_manager.cc:1428] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 06:49:12.274335 5551 heartbeater.cc:503] Master 127.15.232.254:38553 was elected leader, sending a full tablet report...
I20260810 06:49:12.274518 5736 raft_consensus.cc:348] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73fbaaf55700488685538c6f57ddceba" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36203 } } peers { permanent_uuid: "186a313b0be84814b8ee6498852e15ff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36355 } } peers { permanent_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38757 } }
I20260810 06:49:12.274703 5736 raft_consensus.cc:374] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:12.274592 5733 raft_consensus.cc:348] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73fbaaf55700488685538c6f57ddceba" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36203 } } peers { permanent_uuid: "186a313b0be84814b8ee6498852e15ff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36355 } } peers { permanent_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38757 } }
I20260810 06:49:12.274750 5736 raft_consensus.cc:729] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff478ad7b374d8095ee9e04d6ca6c2e, State: Initialized, Role: FOLLOWER
I20260810 06:49:12.274770 5733 raft_consensus.cc:374] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:12.274806 5733 raft_consensus.cc:729] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73fbaaf55700488685538c6f57ddceba, State: Initialized, Role: FOLLOWER
I20260810 06:49:12.274873 5736 consensus_queue.cc:260] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73fbaaf55700488685538c6f57ddceba" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36203 } } peers { permanent_uuid: "186a313b0be84814b8ee6498852e15ff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36355 } } peers { permanent_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38757 } }
I20260810 06:49:12.274957 5733 consensus_queue.cc:260] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73fbaaf55700488685538c6f57ddceba" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36203 } } peers { permanent_uuid: "186a313b0be84814b8ee6498852e15ff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36355 } } peers { permanent_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38757 } }
I20260810 06:49:12.275147 5736 ts_tablet_manager.cc:1428] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:49:12.275267 5717 heartbeater.cc:503] Master 127.15.232.254:38553 was elected leader, sending a full tablet report...
I20260810 06:49:12.275250 5733 ts_tablet_manager.cc:1428] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:49:12.275266 5391 heartbeater.cc:503] Master 127.15.232.254:38553 was elected leader, sending a full tablet report...
W20260810 06:49:12.349371 5554 tablet.cc:2369] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:49:12.454495 5392 tablet.cc:2369] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:49:12.487056 5718 tablet.cc:2369] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:49:12.516768 5741 raft_consensus.cc:482] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:12.516961 5741 raft_consensus.cc:504] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73fbaaf55700488685538c6f57ddceba" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36203 } } peers { permanent_uuid: "186a313b0be84814b8ee6498852e15ff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36355 } } peers { permanent_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38757 } }
I20260810 06:49:12.517457 5741 leader_election.cc:288] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 186a313b0be84814b8ee6498852e15ff (127.15.232.194:36355), 2ff478ad7b374d8095ee9e04d6ca6c2e (127.15.232.195:38757)
I20260810 06:49:12.520915 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.517748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:38853 (local address 127.15.232.194:36355)
0810 06:49:12.517801 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:12.517804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:12.517858 (+ 54us) server_negotiation.cc:407] Connection header received
0810 06:49:12.517996 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:12.517997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:12.518010 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:12.518109 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:12.518301 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.519057 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.520149 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.520527 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.520620 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.520629 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:12.520795 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 06:49:12.520843 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:49:12.520920 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.517986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:57503 (local address 127.15.232.195:38757)
0810 06:49:12.518030 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:12.518033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:12.518041 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:49:12.518075 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:12.518076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:12.518085 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:12.518136 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:12.518369 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.519038 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.520160 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.520534 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.520625 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.520633 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:12.520801 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 06:49:12.520844 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:49:12.520932 5744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.517649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:36355 (local address 127.15.232.193:38853)
0810 06:49:12.517774 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:12.517786 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:12.517902 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:12.518118 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:12.518119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:12.518120 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:12.518204 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.518206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.519167 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.519168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.520048 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.520050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.520620 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.520621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.520694 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.520698 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:49:12.520786 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 06:49:12.520831 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":25,"thread_start_us":72,"threads_started":1}
I20260810 06:49:12.520943 5745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.517836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:38757 (local address 127.15.232.193:57503)
0810 06:49:12.517926 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:12.517938 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:12.518012 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:12.518187 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:12.518188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:12.518190 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:12.518275 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.518277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.519178 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.519179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.520061 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:12.520064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.520625 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.520626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:12.520722 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.520726 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:49:12.520795 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 06:49:12.520844 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":61,"thread_start_us":52,"threads_started":1}
I20260810 06:49:12.521345 5493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbf363d805b4c0f863bf14f7328b192" candidate_uuid: "73fbaaf55700488685538c6f57ddceba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186a313b0be84814b8ee6498852e15ff" is_pre_election: true
I20260810 06:49:12.521528 5493 raft_consensus.cc:2454] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73fbaaf55700488685538c6f57ddceba in term 0.
I20260810 06:49:12.521857 5659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbf363d805b4c0f863bf14f7328b192" candidate_uuid: "73fbaaf55700488685538c6f57ddceba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e" is_pre_election: true
I20260810 06:49:12.521986 5659 raft_consensus.cc:2454] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73fbaaf55700488685538c6f57ddceba in term 0.
I20260810 06:49:12.522188 5256 leader_election.cc:302] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 186a313b0be84814b8ee6498852e15ff, 73fbaaf55700488685538c6f57ddceba; no voters:
I20260810 06:49:12.522389 5741 raft_consensus.cc:2781] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:12.522473 5741 raft_consensus.cc:482] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:12.522511 5741 raft_consensus.cc:3037] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:12.523393 5741 raft_consensus.cc:504] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73fbaaf55700488685538c6f57ddceba" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36203 } } peers { permanent_uuid: "186a313b0be84814b8ee6498852e15ff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36355 } } peers { permanent_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38757 } }
I20260810 06:49:12.523559 5741 leader_election.cc:288] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [CANDIDATE]: Term 1 election: Requested vote from peers 186a313b0be84814b8ee6498852e15ff (127.15.232.194:36355), 2ff478ad7b374d8095ee9e04d6ca6c2e (127.15.232.195:38757)
I20260810 06:49:12.523810 5493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbf363d805b4c0f863bf14f7328b192" candidate_uuid: "73fbaaf55700488685538c6f57ddceba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186a313b0be84814b8ee6498852e15ff"
I20260810 06:49:12.523802 5659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbf363d805b4c0f863bf14f7328b192" candidate_uuid: "73fbaaf55700488685538c6f57ddceba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e"
I20260810 06:49:12.523902 5659 raft_consensus.cc:3037] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:12.523902 5493 raft_consensus.cc:3037] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:12.525068 5493 raft_consensus.cc:2454] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73fbaaf55700488685538c6f57ddceba in term 1.
I20260810 06:49:12.525139 5659 raft_consensus.cc:2454] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73fbaaf55700488685538c6f57ddceba in term 1.
I20260810 06:49:12.525405 5256 leader_election.cc:302] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff478ad7b374d8095ee9e04d6ca6c2e, 73fbaaf55700488685538c6f57ddceba; no voters:
I20260810 06:49:12.525524 5741 raft_consensus.cc:2781] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:12.525733 5741 raft_consensus.cc:686] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 1 LEADER]: Becoming Leader. State: Replica: 73fbaaf55700488685538c6f57ddceba, State: Running, Role: LEADER
I20260810 06:49:12.525852 5741 consensus_queue.cc:237] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73fbaaf55700488685538c6f57ddceba" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36203 } } peers { permanent_uuid: "186a313b0be84814b8ee6498852e15ff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36355 } } peers { permanent_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38757 } }
I20260810 06:49:12.526873 5158 catalog_manager.cc:5495] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba reported cstate change: term changed from 0 to 1, leader changed from <none> to 73fbaaf55700488685538c6f57ddceba (127.15.232.193). New cstate: current_term: 1 leader_uuid: "73fbaaf55700488685538c6f57ddceba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73fbaaf55700488685538c6f57ddceba" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 36203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "186a313b0be84814b8ee6498852e15ff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38757 } health_report { overall_health: UNKNOWN } } }
I20260810 06:49:12.554991 5729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.552411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45974 (local address 127.15.232.193:36203)
0810 06:49:12.552479 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:12.552482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:12.552491 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:49:12.552534 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:12.552535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:12.552566 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:12.552647 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:12.552776 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.553388 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.554177 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:12.554386 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:12.554464 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:12.554619 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:12.554723 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:12.554831 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:12.554891 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 06:49:12.554903 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:49:12.557060 5659 raft_consensus.cc:1261] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e [term 1 FOLLOWER]: Refusing update from remote peer 73fbaaf55700488685538c6f57ddceba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:12.557090 5493 raft_consensus.cc:1261] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff [term 1 FOLLOWER]: Refusing update from remote peer 73fbaaf55700488685538c6f57ddceba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:12.557518 5741 consensus_queue.cc:1035] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff478ad7b374d8095ee9e04d6ca6c2e" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:12.557698 5746 consensus_queue.cc:1035] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [LEADER]: Connected to new peer: Peer: permanent_uuid: "186a313b0be84814b8ee6498852e15ff" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 36355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:12.560413 5749 mvcc.cc:205] Tried to move back new op lower bound from 7316867287270682624 to 7316867287146672128. Current Snapshot: MvccSnapshot[applied={T|T < 7316867287270682624}]
I20260810 06:49:12.563246 5157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:12.563418 5157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:12.564582 5157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fc542a1bd9143fbae7754f483d9d6b0: Sending DeleteTablet for 3 replicas of tablet cfbf363d805b4c0f863bf14f7328b192
I20260810 06:49:12.565200 5313 tablet_service.cc:1514] Processing DeleteTablet for tablet cfbf363d805b4c0f863bf14f7328b192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:12 UTC) from {username='slave'} at 127.0.0.1:45960
I20260810 06:49:12.565308 5633 tablet_service.cc:1514] Processing DeleteTablet for tablet cfbf363d805b4c0f863bf14f7328b192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:12 UTC) from {username='slave'} at 127.0.0.1:37478
I20260810 06:49:12.565552 5759 tablet_replica.cc:330] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba: stopping tablet replica
I20260810 06:49:12.565552 5471 tablet_service.cc:1514] Processing DeleteTablet for tablet cfbf363d805b4c0f863bf14f7328b192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:12 UTC) from {username='slave'} at 127.0.0.1:44362
I20260810 06:49:12.565680 5759 raft_consensus.cc:2229] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:49:12.565826 5759 raft_consensus.cc:2258] T cfbf363d805b4c0f863bf14f7328b192 P 73fbaaf55700488685538c6f57ddceba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:12.565836 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 5234
I20260810 06:49:12.570067 5761 tablet_replica.cc:330] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff: stopping tablet replica
I20260810 06:49:12.570282 5761 raft_consensus.cc:2229] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:12.570524 5761 raft_consensus.cc:2258] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:12.572522 5761 ts_tablet_manager.cc:1905] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:12.574621 5761 ts_tablet_manager.cc:1918] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:12.574782 5761 log.cc:1196] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/cfbf363d805b4c0f863bf14f7328b192
I20260810 06:49:12.575134 5761 ts_tablet_manager.cc:1939] T cfbf363d805b4c0f863bf14f7328b192 P 186a313b0be84814b8ee6498852e15ff: Deleting consensus metadata
I20260810 06:49:12.578238 5142 catalog_manager.cc:4841] TS 186a313b0be84814b8ee6498852e15ff (127.15.232.194:36355): tablet cfbf363d805b4c0f863bf14f7328b192 (table test-table [id=e25b2fbe53db49c9bf881719661982cb]) successfully deleted
I20260810 06:49:12.578596 5760 tablet_replica.cc:330] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e: stopping tablet replica
I20260810 06:49:12.578778 5760 raft_consensus.cc:2229] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:12.578941 5760 raft_consensus.cc:2258] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:12.579450 5760 ts_tablet_manager.cc:1905] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:12.581063 5760 ts_tablet_manager.cc:1918] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:12.581197 5760 log.cc:1196] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/cfbf363d805b4c0f863bf14f7328b192
I20260810 06:49:12.581449 5760 ts_tablet_manager.cc:1939] T cfbf363d805b4c0f863bf14f7328b192 P 2ff478ad7b374d8095ee9e04d6ca6c2e: Deleting consensus metadata
I20260810 06:49:12.582060 5142 catalog_manager.cc:4841] TS 2ff478ad7b374d8095ee9e04d6ca6c2e (127.15.232.195:38757): tablet cfbf363d805b4c0f863bf14f7328b192 (table test-table [id=e25b2fbe53db49c9bf881719661982cb]) successfully deleted
W20260810 06:49:12.582662 5145 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:36203 (error 108)
I20260810 06:49:12.583201 5730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:12.583054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:36203 (local address 127.0.0.1:45978)
0810 06:49:12.583110 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:12.583158 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:36203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 06:49:12.583209 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 5394
W20260810 06:49:12.585300 5145 catalog_manager.cc:4584] TS 73fbaaf55700488685538c6f57ddceba (127.15.232.193:36203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cfbf363d805b4c0f863bf14f7328b192: Network error: Client connection negotiation failed: client connection to 127.15.232.193:36203: connect: Connection refused (error 111)
I20260810 06:49:12.594033 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 5556
I20260810 06:49:12.602869 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 5121
2026-08-10T06:49:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (969 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:12 dist-test-slave-vxj2 krb5kdc[5767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:12 dist-test-slave-vxj2 krb5kdc[5767](info): set up 2 sockets
Aug 10 06:49:12 dist-test-slave-vxj2 krb5kdc[5767](info): commencing operation
krb5kdc: starting...
W20260810 06:49:13.743685 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:13 dist-test-slave-vxj2 krb5kdc[5767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:49:13.874552 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:44479
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:44479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:13.960736 5782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:13.960985 5782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:13.961011 5782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:13.962529 5782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:13.962618 5782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:13.962635 5782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:13.962649 5782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:13.962662 5782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:13.964406 5782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:44479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:44479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:13.964668 5782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:13.964892 5782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:13.970531 5793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:13.970525 5787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:13.970525 5789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:13.970906 5782 server_base.cc:1034] running on GCE node
I20260810 06:49:13.971216 5782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:13.971582 5782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:13.972761 5782 hybrid_clock.cc:648] HybridClock initialized: now 1786344553972750 us; error 50 us; skew 500 ppm
Aug 10 06:49:13 dist-test-slave-vxj2 krb5kdc[5767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344553, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:13.975531 5782 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:49:13.976048 5782 webserver.cc:460] Webserver started at http://127.15.232.254:37305/ using document root <none> and password file <none>
I20260810 06:49:13.976378 5782 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:13.976449 5782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:13.976593 5782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:13.977831 5782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "4d595b556e7e49b2b8c68ee354e0f3ef"
format_stamp: "Formatted at 2026-08-10 06:49:13 on dist-test-slave-vxj2"
server_key: "a50eef09d232cd54c4a7447a0dbcf705"
server_key_iv: "70ffefa11695fe2d389ace2d17159ec0"
server_key_version: "encryptionkey@0"
I20260810 06:49:13.978238 5782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "4d595b556e7e49b2b8c68ee354e0f3ef"
format_stamp: "Formatted at 2026-08-10 06:49:13 on dist-test-slave-vxj2"
server_key: "a50eef09d232cd54c4a7447a0dbcf705"
server_key_iv: "70ffefa11695fe2d389ace2d17159ec0"
server_key_version: "encryptionkey@0"
I20260810 06:49:13.979687 5782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:49:13.980572 5800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:13.980783 5782 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:13.980933 5782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "4d595b556e7e49b2b8c68ee354e0f3ef"
format_stamp: "Formatted at 2026-08-10 06:49:13 on dist-test-slave-vxj2"
server_key: "a50eef09d232cd54c4a7447a0dbcf705"
server_key_iv: "70ffefa11695fe2d389ace2d17159ec0"
server_key_version: "encryptionkey@0"
I20260810 06:49:13.981019 5782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:14.005275 5782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:14.007696 5782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:14.007932 5782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:14.012353 5782 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:44479
I20260810 06:49:14.012398 5860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:44479 every 8 connection(s)
I20260810 06:49:14.012778 5782 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:14.013797 5861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:14.016332 5861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef: Bootstrap starting.
I20260810 06:49:14.016969 5861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:14.017311 5861 log.cc:824] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:14.018100 5861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef: No bootstrap required, opened a new log
I20260810 06:49:14.019716 5861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d595b556e7e49b2b8c68ee354e0f3ef" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44479 } }
I20260810 06:49:14.019872 5861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:14.019896 5861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d595b556e7e49b2b8c68ee354e0f3ef, State: Initialized, Role: FOLLOWER
I20260810 06:49:14.020074 5861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d595b556e7e49b2b8c68ee354e0f3ef" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44479 } }
I20260810 06:49:14.020141 5861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:14.020184 5861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:14.020236 5861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:14.020691 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 5782
I20260810 06:49:14.020828 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:14.021042 16291 external_mini_cluster.cc:1428] Setting key 8f24c523f818e77eee8d6e502796dd2f
I20260810 06:49:14.021039 5861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d595b556e7e49b2b8c68ee354e0f3ef" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44479 } }
I20260810 06:49:14.021173 5861 leader_election.cc:302] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d595b556e7e49b2b8c68ee354e0f3ef; no voters:
I20260810 06:49:14.021363 5861 leader_election.cc:288] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:14.021406 5864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:14.021559 5864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [term 1 LEADER]: Becoming Leader. State: Replica: 4d595b556e7e49b2b8c68ee354e0f3ef, State: Running, Role: LEADER
I20260810 06:49:14.021672 5861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:14.021659 5864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d595b556e7e49b2b8c68ee354e0f3ef" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44479 } }
I20260810 06:49:14.022042 5865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d595b556e7e49b2b8c68ee354e0f3ef. Latest consensus state: current_term: 1 leader_uuid: "4d595b556e7e49b2b8c68ee354e0f3ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d595b556e7e49b2b8c68ee354e0f3ef" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44479 } } }
I20260810 06:49:14.022132 5865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:14.022186 5866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d595b556e7e49b2b8c68ee354e0f3ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d595b556e7e49b2b8c68ee354e0f3ef" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 44479 } } }
I20260810 06:49:14.022256 5866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:14.022393 5873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:14.022971 5873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:14.024714 5873 catalog_manager.cc:1347] Generated new cluster ID: f32e2c4025e44040b32f5953219123df
I20260810 06:49:14.024771 5873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:49:14 dist-test-slave-vxj2 krb5kdc[5767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:14.034593 5868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.021638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40062 (local address 127.15.232.254:44479)
0810 06:49:14.022062 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:14.022072 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:14.022110 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:49:14.022445 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:14.022455 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:14.022527 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:14.022693 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:14.023522 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.024923 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.026019 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.026699 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.026844 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.029851 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:14.029856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:14.029856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:14.029871 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:14.032670 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:14.033167 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:14.033170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:14.033171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:14.033305 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:14.033548 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:14.033549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:14.033550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:14.033886 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:14.034044 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:14.034191 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 06:49:14.034338 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":94,"threads_started":1}
I20260810 06:49:14.039306 5873 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:14.040073 5873 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:14.047209 5873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef: Generated new TSK 0
I20260810 06:49:14.047511 5873 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
Entry for principal kudu/127.15.232.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab.
I20260810 06:49:14.084251 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44479
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:14.170825 5890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:14.171058 5890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:14.171087 5890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:14.172318 5890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:14.172367 5890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:14.172396 5890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:49:14.174031 5890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.193.keytab
--principal=kudu/127.15.232.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:14.174309 5890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:14.174598 5890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:14.180418 5900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:14.180469 5897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:14.180449 5898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:14.180744 5890 server_base.cc:1034] running on GCE node
I20260810 06:49:14.180960 5890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:14.181237 5890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:14.182432 5890 hybrid_clock.cc:648] HybridClock initialized: now 1786344554182417 us; error 29 us; skew 500 ppm
Aug 10 06:49:14 dist-test-slave-vxj2 krb5kdc[5767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344554, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:14.185525 5890 init.cc:377] Logged in from keytab as kudu/127.15.232.193@KRBTEST.COM (short username kudu)
I20260810 06:49:14.186064 5890 webserver.cc:460] Webserver started at http://127.15.232.193:37273/ using document root <none> and password file <none>
I20260810 06:49:14.186322 5890 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:14.186396 5890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:14.186578 5890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:14.187808 5890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "4d66a9db57b74339813c7648d7a1d423"
format_stamp: "Formatted at 2026-08-10 06:49:14 on dist-test-slave-vxj2"
server_key: "7654011f70722d9bfd02f8f2c2239809"
server_key_iv: "da5870380454b9157b3c5fd0eec02972"
server_key_version: "encryptionkey@0"
I20260810 06:49:14.188216 5890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "4d66a9db57b74339813c7648d7a1d423"
format_stamp: "Formatted at 2026-08-10 06:49:14 on dist-test-slave-vxj2"
server_key: "7654011f70722d9bfd02f8f2c2239809"
server_key_iv: "da5870380454b9157b3c5fd0eec02972"
server_key_version: "encryptionkey@0"
I20260810 06:49:14.190076 5890 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260810 06:49:14.191074 5907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:14.191280 5890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:49:14.191447 5890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "4d66a9db57b74339813c7648d7a1d423"
format_stamp: "Formatted at 2026-08-10 06:49:14 on dist-test-slave-vxj2"
server_key: "7654011f70722d9bfd02f8f2c2239809"
server_key_iv: "da5870380454b9157b3c5fd0eec02972"
server_key_version: "encryptionkey@0"
I20260810 06:49:14.191541 5890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:14.205778 5890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:14.208467 5890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:14.208654 5890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:14.208931 5890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:14.209369 5890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:14.209425 5890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:14.209478 5890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:14.209506 5890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:14.215493 5890 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:39843
I20260810 06:49:14.215547 6038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:39843 every 8 connection(s)
I20260810 06:49:14.215916 5890 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:49:14.220625 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 5890
I20260810 06:49:14.220784 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:49:14.221038 16291 external_mini_cluster.cc:1428] Setting key 5c7e2b355a5807b1d728d2d8e809b223
Aug 10 06:49:14 dist-test-slave-vxj2 krb5kdc[5767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344554, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.193@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:14.227324 5868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.216717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:34515 (local address 127.15.232.254:44479)
0810 06:49:14.216768 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:14.216771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:14.217072 (+ 301us) server_negotiation.cc:407] Connection header received
0810 06:49:14.217647 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:14.217648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:14.217664 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:14.217714 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:14.218775 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.219301 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.220394 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.220607 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.220700 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.223582 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:14.223584 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:14.223585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:14.223594 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:14.225258 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:14.225854 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:14.225856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:14.225856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:14.225975 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:14.226314 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:14.226316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:14.226316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:14.226488 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:14.226584 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:14.227001 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 06:49:14.227172 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:49:14.227620 6042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.216677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44479 (local address 127.15.232.193:34515)
0810 06:49:14.216933 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:14.216976 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:14.217531 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:14.217818 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:14.217822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:14.218433 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:14.218657 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.218662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.219424 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.219425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.220285 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.220288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.220713 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.220714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.220892 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.221621 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:14.221630 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:14.223465 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:14.225365 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:14.225374 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:14.225377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:14.225744 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:14.226076 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:14.226077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:14.226077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:14.226227 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:14.226574 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:14.226583 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:14.226821 (+ 238us) client_negotiation.cc:769] Sending connection context
0810 06:49:14.226952 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 06:49:14.227263 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":102,"threads_started":1}
I20260810 06:49:14.228297 6039 heartbeater.cc:347] Connected to a master server at 127.15.232.254:44479
I20260810 06:49:14.228442 6039 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:14.228677 6039 heartbeater.cc:511] Master 127.15.232.254:44479 requested a full tablet report, sending...
I20260810 06:49:14.229417 5818 ts_manager.cc:194] Registered new tserver with Master: 4d66a9db57b74339813c7648d7a1d423 (127.15.232.193:39843)
I20260810 06:49:14.230429 5818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.193@KRBTEST.COM'} at 127.15.232.193:34515
WARNING: no policy specified for kudu/127.15.232.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
Entry for principal kudu/127.15.232.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab.
I20260810 06:49:14.252559 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44479
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:14.340222 6046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:14.340477 6046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:14.340505 6046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:14.341799 6046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:14.341851 6046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:14.341881 6046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:49:14.343415 6046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.194.keytab
--principal=kudu/127.15.232.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:14.343640 6046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:14.343847 6046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:14.349418 6057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:14.349368 6053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:14.349543 6052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:14.349545 6046 server_base.cc:1034] running on GCE node
I20260810 06:49:14.349889 6046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:14.350183 6046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:14.351439 6046 hybrid_clock.cc:648] HybridClock initialized: now 1786344554351434 us; error 45 us; skew 500 ppm
Aug 10 06:49:14 dist-test-slave-vxj2 krb5kdc[5767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344554, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:14.354385 6046 init.cc:377] Logged in from keytab as kudu/127.15.232.194@KRBTEST.COM (short username kudu)
I20260810 06:49:14.354826 6046 webserver.cc:460] Webserver started at http://127.15.232.194:35357/ using document root <none> and password file <none>
I20260810 06:49:14.355113 6046 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:14.355166 6046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:14.355321 6046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:14.356462 6046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "70e9824e4c9d4eafa451f8e0df057cf4"
format_stamp: "Formatted at 2026-08-10 06:49:14 on dist-test-slave-vxj2"
server_key: "c6701d958f4e139e5246b2d141ae4ebc"
server_key_iv: "32990f00701a71ecf7cec4a5a373f8c1"
server_key_version: "encryptionkey@0"
I20260810 06:49:14.356858 6046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "70e9824e4c9d4eafa451f8e0df057cf4"
format_stamp: "Formatted at 2026-08-10 06:49:14 on dist-test-slave-vxj2"
server_key: "c6701d958f4e139e5246b2d141ae4ebc"
server_key_iv: "32990f00701a71ecf7cec4a5a373f8c1"
server_key_version: "encryptionkey@0"
I20260810 06:49:14.358387 6046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:14.359283 6065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:14.359515 6046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:14.359635 6046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "70e9824e4c9d4eafa451f8e0df057cf4"
format_stamp: "Formatted at 2026-08-10 06:49:14 on dist-test-slave-vxj2"
server_key: "c6701d958f4e139e5246b2d141ae4ebc"
server_key_iv: "32990f00701a71ecf7cec4a5a373f8c1"
server_key_version: "encryptionkey@0"
I20260810 06:49:14.359723 6046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:14.385769 6046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:14.388387 6046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:14.388593 6046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:14.388865 6046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:14.389274 6046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:14.389343 6046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:14.389393 6046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:14.389422 6046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:14.396020 6046 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:39833
I20260810 06:49:14.396111 6198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:39833 every 8 connection(s)
I20260810 06:49:14.396484 6046 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:49:14.398756 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 6046
I20260810 06:49:14.398908 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:49:14.399115 16291 external_mini_cluster.cc:1428] Setting key ec5a37bfa56439b4786c98fb6b846496
Aug 10 06:49:14 dist-test-slave-vxj2 krb5kdc[5767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344554, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.194@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:14.407975 5868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.397392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:60459 (local address 127.15.232.254:44479)
0810 06:49:14.397442 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:14.397445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:14.397663 (+ 218us) server_negotiation.cc:407] Connection header received
0810 06:49:14.398317 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:14.398319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:14.398332 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:14.398402 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:14.399582 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.400175 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.401266 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.401495 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.401578 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.404462 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:14.404464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:14.404465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:14.404475 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:14.406144 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:14.406667 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:14.406669 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:14.406670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:14.406745 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:14.407089 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:14.407090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:14.407091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:14.407227 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:14.407317 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:14.407759 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 06:49:14.407873 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:49:14.408233 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.397348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44479 (local address 127.15.232.194:60459)
0810 06:49:14.397529 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:14.397568 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:14.398170 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:14.398479 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:14.398484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:14.399196 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:14.399483 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.399490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.400282 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.400283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.401134 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.401137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.401589 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.401591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.401743 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.402375 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:14.402385 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:14.404313 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:14.406257 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:14.406269 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:14.406272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:14.406576 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:14.406818 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:14.406819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:14.406819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:14.407012 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:14.407308 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:14.407317 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:14.407568 (+ 251us) client_negotiation.cc:769] Sending connection context
0810 06:49:14.407727 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 06:49:14.407943 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.15.232.195@KRBTEST.COM; defaulting to no policy
I20260810 06:49:14.408761 6200 heartbeater.cc:347] Connected to a master server at 127.15.232.254:44479
I20260810 06:49:14.408900 6200 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:14.409103 6200 heartbeater.cc:511] Master 127.15.232.254:44479 requested a full tablet report, sending...
I20260810 06:49:14.409688 5818 ts_manager.cc:194] Registered new tserver with Master: 70e9824e4c9d4eafa451f8e0df057cf4 (127.15.232.194:39833)
I20260810 06:49:14.410100 5818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.194@KRBTEST.COM'} at 127.15.232.194:60459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
Entry for principal kudu/127.15.232.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab.
I20260810 06:49:14.428750 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44479
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:14.517580 6205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:14.517882 6205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:14.517930 6205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:14.519177 6205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:14.519224 6205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:14.519251 6205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:49:14.520736 6205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.195.keytab
--principal=kudu/127.15.232.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:44479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:14.520967 6205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:14.521273 6205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:14.529040 6216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:14.529192 6205 server_base.cc:1034] running on GCE node
W20260810 06:49:14.529115 6211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:14.528990 6210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:14.529635 6205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:14.530020 6205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:14.531229 6205 hybrid_clock.cc:648] HybridClock initialized: now 1786344554531203 us; error 46 us; skew 500 ppm
Aug 10 06:49:14 dist-test-slave-vxj2 krb5kdc[5767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344554, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:14.534240 6205 init.cc:377] Logged in from keytab as kudu/127.15.232.195@KRBTEST.COM (short username kudu)
I20260810 06:49:14.534904 6205 webserver.cc:460] Webserver started at http://127.15.232.195:35239/ using document root <none> and password file <none>
I20260810 06:49:14.535169 6205 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:14.535249 6205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:14.535418 6205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:14.536613 6205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "060dcfdb56cd4585aa5bb9bb5e055dff"
format_stamp: "Formatted at 2026-08-10 06:49:14 on dist-test-slave-vxj2"
server_key: "027208e6c723258a5e025b0f0d480722"
server_key_iv: "2629b70a5d3c0dd6464fd52b587736a1"
server_key_version: "encryptionkey@0"
I20260810 06:49:14.537019 6205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "060dcfdb56cd4585aa5bb9bb5e055dff"
format_stamp: "Formatted at 2026-08-10 06:49:14 on dist-test-slave-vxj2"
server_key: "027208e6c723258a5e025b0f0d480722"
server_key_iv: "2629b70a5d3c0dd6464fd52b587736a1"
server_key_version: "encryptionkey@0"
I20260810 06:49:14.538663 6205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:49:14.539695 6224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:14.539928 6205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:14.540059 6205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "060dcfdb56cd4585aa5bb9bb5e055dff"
format_stamp: "Formatted at 2026-08-10 06:49:14 on dist-test-slave-vxj2"
server_key: "027208e6c723258a5e025b0f0d480722"
server_key_iv: "2629b70a5d3c0dd6464fd52b587736a1"
server_key_version: "encryptionkey@0"
I20260810 06:49:14.540148 6205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:14.583391 6205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:14.586114 6205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:14.586278 6205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:14.586557 6205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:14.587030 6205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:14.587070 6205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:14.587093 6205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:14.587106 6205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:14.593371 6205 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:45207
I20260810 06:49:14.593464 6358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:45207 every 8 connection(s)
I20260810 06:49:14.593838 6205 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:49:14.595956 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 6205
I20260810 06:49:14.596076 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:49:14.596271 16291 external_mini_cluster.cc:1428] Setting key 285822cced090fa07428712527622d08
Aug 10 06:49:14 dist-test-slave-vxj2 krb5kdc[5767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344554, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.195@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:14.604935 5868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.594854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:42099 (local address 127.15.232.254:44479)
0810 06:49:14.594944 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:14.594947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:14.595134 (+ 187us) server_negotiation.cc:407] Connection header received
0810 06:49:14.595681 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:14.595683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:14.595694 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:14.595771 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:14.597231 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.597762 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.598960 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.599153 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.599215 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.601684 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:14.601686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:14.601686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:14.601694 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:14.603249 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:14.603833 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:14.603834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:14.603834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:14.603926 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:14.604249 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:14.604250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:14.604250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:14.604371 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:14.604433 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:14.604751 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 06:49:14.604825 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:49:14.605113 6361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.594785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:44479 (local address 127.15.232.195:42099)
0810 06:49:14.594962 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:14.595008 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:14.595583 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:14.595872 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:14.595878 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:14.596786 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:14.597093 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.597099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.597928 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.597929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.598853 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.598866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.599243 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.599244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.599363 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.599947 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:14.599957 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:14.601577 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:14.603362 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:14.603373 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:14.603375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:14.603678 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:14.604025 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:14.604026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:14.604026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:14.604156 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:14.604458 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:14.604465 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:14.604619 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 06:49:14.604720 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 06:49:14.604868 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260810 06:49:14.605737 6359 heartbeater.cc:347] Connected to a master server at 127.15.232.254:44479
I20260810 06:49:14.605870 6359 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:14.606096 6359 heartbeater.cc:511] Master 127.15.232.254:44479 requested a full tablet report, sending...
I20260810 06:49:14.606621 5818 ts_manager.cc:194] Registered new tserver with Master: 060dcfdb56cd4585aa5bb9bb5e055dff (127.15.232.195:45207)
I20260810 06:49:14.607038 5818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.195@KRBTEST.COM'} at 127.15.232.195:42099
I20260810 06:49:14.608261 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:49:14.616132 5868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.609668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32848 (local address 127.15.232.254:44479)
0810 06:49:14.609740 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:14.609743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:14.609887 (+ 144us) server_negotiation.cc:407] Connection header received
0810 06:49:14.609916 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:14.609916 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:14.609926 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:14.609984 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:14.610560 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.611113 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.611866 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.612062 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.612116 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.613546 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:14.613547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:14.613548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:14.613555 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:14.614865 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:14.615205 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:14.615206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:14.615206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:14.615268 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:14.615508 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:14.615509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:14.615509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:14.615667 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:14.615750 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:14.615928 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 06:49:14.616002 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:49:14.618064 5818 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:49:14.619647 5818 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:14.627746 6379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.622917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57378 (local address 127.15.232.193:39843)
0810 06:49:14.623304 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:14.623310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:14.623322 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:49:14.623365 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:14.623368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:14.623385 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:14.623463 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:14.623625 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.624916 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.626305 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.627158 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.627250 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.627313 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:14.627615 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 06:49:14.627657 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":99,"threads_started":1}
I20260810 06:49:14.627737 6378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.622554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:45207 (local address 127.0.0.1:46694)
0810 06:49:14.622928 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:14.622978 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:14.623094 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:14.624128 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:14.624129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:14.624131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:14.624169 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.624170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.625568 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.625569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.626651 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.626653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.627448 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.627449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.627507 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.627511 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:49:14.627561 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 06:49:14.627633 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":112,"threads_started":1}
I20260810 06:49:14.627969 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.622661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57050 (local address 127.15.232.194:39833)
0810 06:49:14.623618 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:14.623622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:14.623633 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:49:14.623680 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:14.623682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:14.623700 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:14.623792 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:14.624027 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.624849 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.626578 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.627351 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.627467 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.627524 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:14.627785 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 06:49:14.627838 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":891,"thread_start_us":93,"threads_started":1}
I20260810 06:49:14.627993 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.622763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:39843 (local address 127.0.0.1:57378)
0810 06:49:14.622984 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:14.622994 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:14.623056 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:14.623471 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:14.623475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:14.623477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:14.623539 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.623540 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.625047 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.625048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.626191 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.626195 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.627272 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.627273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.627359 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.627375 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 06:49:14.627878 (+ 503us) client_negotiation.cc:240] Negotiation successful
0810 06:49:14.627929 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":121,"thread_start_us":88,"threads_started":1}
I20260810 06:49:14.628120 6374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.622547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:39833 (local address 127.0.0.1:57050)
0810 06:49:14.622928 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:14.622980 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:14.623056 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:14.623852 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:14.623853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:14.623856 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:14.623912 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.623914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.624963 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.624964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.626441 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.626444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.627466 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.627467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.627540 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.627545 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:49:14.627602 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 06:49:14.627652 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":39,"thread_start_us":56,"threads_started":1}
I20260810 06:49:14.628129 6377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.622663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46694 (local address 127.15.232.195:45207)
0810 06:49:14.623963 (+ 1300us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:14.623968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:14.623979 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:49:14.624019 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:14.624021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:14.624037 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:14.624122 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:14.624239 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.625456 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.626739 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.627282 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.627357 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.627412 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:14.627973 (+ 561us) server_negotiation.cc:299] Negotiation successful
0810 06:49:14.628024 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1256,"thread_start_us":112,"threads_started":1}
I20260810 06:49:14.628824 5964 tablet_service.cc:1467] Processing CreateTablet for tablet 2f65e0e806af4666877f19d0e76bc532 (DEFAULT_TABLE table=test-table [id=f18138d7b6e542dd84669ad5745e11be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:14.628841 6120 tablet_service.cc:1467] Processing CreateTablet for tablet 2f65e0e806af4666877f19d0e76bc532 (DEFAULT_TABLE table=test-table [id=f18138d7b6e542dd84669ad5745e11be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:14.628836 6278 tablet_service.cc:1467] Processing CreateTablet for tablet 2f65e0e806af4666877f19d0e76bc532 (DEFAULT_TABLE table=test-table [id=f18138d7b6e542dd84669ad5745e11be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:14.629256 6278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f65e0e806af4666877f19d0e76bc532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:14.629266 6120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f65e0e806af4666877f19d0e76bc532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:14.629276 5964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f65e0e806af4666877f19d0e76bc532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:14.632192 6380 tablet_bootstrap.cc:492] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423: Bootstrap starting.
I20260810 06:49:14.633039 6380 tablet_bootstrap.cc:654] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:14.633318 6382 tablet_bootstrap.cc:492] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff: Bootstrap starting.
I20260810 06:49:14.633658 6380 log.cc:824] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:14.634001 6382 tablet_bootstrap.cc:654] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:14.634479 6380 tablet_bootstrap.cc:492] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423: No bootstrap required, opened a new log
I20260810 06:49:14.634541 6380 ts_tablet_manager.cc:1397] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 06:49:14.634548 6382 log.cc:824] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:14.634928 6381 tablet_bootstrap.cc:492] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4: Bootstrap starting.
I20260810 06:49:14.635406 6382 tablet_bootstrap.cc:492] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff: No bootstrap required, opened a new log
I20260810 06:49:14.635480 6382 ts_tablet_manager.cc:1397] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:49:14.635617 6381 tablet_bootstrap.cc:654] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:14.636209 6381 log.cc:824] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:14.636556 6380 raft_consensus.cc:348] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e9824e4c9d4eafa451f8e0df057cf4" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39833 } } peers { permanent_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45207 } } peers { permanent_uuid: "4d66a9db57b74339813c7648d7a1d423" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 39843 } }
I20260810 06:49:14.636775 6380 raft_consensus.cc:374] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:14.636821 6380 raft_consensus.cc:729] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d66a9db57b74339813c7648d7a1d423, State: Initialized, Role: FOLLOWER
I20260810 06:49:14.636978 6380 consensus_queue.cc:260] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e9824e4c9d4eafa451f8e0df057cf4" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39833 } } peers { permanent_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45207 } } peers { permanent_uuid: "4d66a9db57b74339813c7648d7a1d423" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 39843 } }
I20260810 06:49:14.637043 6381 tablet_bootstrap.cc:492] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4: No bootstrap required, opened a new log
I20260810 06:49:14.637110 6381 ts_tablet_manager.cc:1397] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:49:14.637388 6039 heartbeater.cc:503] Master 127.15.232.254:44479 was elected leader, sending a full tablet report...
I20260810 06:49:14.637379 6380 ts_tablet_manager.cc:1428] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:49:14.637902 6382 raft_consensus.cc:348] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e9824e4c9d4eafa451f8e0df057cf4" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39833 } } peers { permanent_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45207 } } peers { permanent_uuid: "4d66a9db57b74339813c7648d7a1d423" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 39843 } }
I20260810 06:49:14.638072 6382 raft_consensus.cc:374] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:14.638105 6382 raft_consensus.cc:729] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 060dcfdb56cd4585aa5bb9bb5e055dff, State: Initialized, Role: FOLLOWER
I20260810 06:49:14.638218 6382 consensus_queue.cc:260] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e9824e4c9d4eafa451f8e0df057cf4" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39833 } } peers { permanent_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45207 } } peers { permanent_uuid: "4d66a9db57b74339813c7648d7a1d423" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 39843 } }
I20260810 06:49:14.638533 6382 ts_tablet_manager.cc:1428] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:49:14.638566 6359 heartbeater.cc:503] Master 127.15.232.254:44479 was elected leader, sending a full tablet report...
I20260810 06:49:14.639370 6381 raft_consensus.cc:348] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e9824e4c9d4eafa451f8e0df057cf4" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39833 } } peers { permanent_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45207 } } peers { permanent_uuid: "4d66a9db57b74339813c7648d7a1d423" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 39843 } }
I20260810 06:49:14.639592 6381 raft_consensus.cc:374] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:14.639648 6381 raft_consensus.cc:729] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e9824e4c9d4eafa451f8e0df057cf4, State: Initialized, Role: FOLLOWER
I20260810 06:49:14.639780 6381 consensus_queue.cc:260] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e9824e4c9d4eafa451f8e0df057cf4" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39833 } } peers { permanent_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45207 } } peers { permanent_uuid: "4d66a9db57b74339813c7648d7a1d423" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 39843 } }
I20260810 06:49:14.640049 6381 ts_tablet_manager.cc:1428] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:49:14.640101 6200 heartbeater.cc:503] Master 127.15.232.254:44479 was elected leader, sending a full tablet report...
W20260810 06:49:14.647104 6201 tablet.cc:2369] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:49:14.716825 6041 tablet.cc:2369] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:49:14.844573 6360 tablet.cc:2369] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:49:14.880085 6388 raft_consensus.cc:482] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:14.880228 6388 raft_consensus.cc:504] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e9824e4c9d4eafa451f8e0df057cf4" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39833 } } peers { permanent_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45207 } } peers { permanent_uuid: "4d66a9db57b74339813c7648d7a1d423" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 39843 } }
I20260810 06:49:14.880685 6388 leader_election.cc:288] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70e9824e4c9d4eafa451f8e0df057cf4 (127.15.232.194:39833), 060dcfdb56cd4585aa5bb9bb5e055dff (127.15.232.195:45207)
I20260810 06:49:14.884047 6377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.881001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:49995 (local address 127.15.232.195:45207)
0810 06:49:14.881053 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:14.881055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:14.881190 (+ 135us) server_negotiation.cc:407] Connection header received
0810 06:49:14.881291 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:14.881292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:14.881304 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:14.881418 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:14.881612 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.882076 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.883172 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.883663 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.883753 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.883763 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:14.883919 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 06:49:14.883963 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:49:14.884055 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.880996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:37745 (local address 127.15.232.194:39833)
0810 06:49:14.881049 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:14.881052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:14.881208 (+ 156us) server_negotiation.cc:407] Connection header received
0810 06:49:14.881324 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:14.881325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:14.881340 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:14.881407 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:14.881590 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.882075 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.883179 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.883700 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.883778 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.883787 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:14.883941 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 06:49:14.883979 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:49:14.884069 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.880933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:45207 (local address 127.15.232.193:49995)
0810 06:49:14.881089 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:14.881099 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:14.881218 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:14.881442 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:14.881443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:14.881445 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:14.881536 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.881538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.882174 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.882175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.883067 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.883070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.883752 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.883753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.883827 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.883832 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:49:14.883910 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:49:14.883960 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":42,"thread_start_us":93,"threads_started":1}
I20260810 06:49:14.884085 6391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.880919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:39833 (local address 127.15.232.193:37745)
0810 06:49:14.881083 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:14.881094 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:14.881230 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:14.881422 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:14.881423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:14.881425 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:14.881512 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.881514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.882166 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.882167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.883068 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:14.883070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.883778 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.883779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:14.883857 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.883861 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:49:14.883934 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:49:14.883979 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":4,"thread_start_us":96,"threads_started":1}
I20260810 06:49:14.884444 6304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f65e0e806af4666877f19d0e76bc532" candidate_uuid: "4d66a9db57b74339813c7648d7a1d423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff" is_pre_election: true
I20260810 06:49:14.884639 6304 raft_consensus.cc:2454] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d66a9db57b74339813c7648d7a1d423 in term 0.
I20260810 06:49:14.884719 6141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f65e0e806af4666877f19d0e76bc532" candidate_uuid: "4d66a9db57b74339813c7648d7a1d423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e9824e4c9d4eafa451f8e0df057cf4" is_pre_election: true
I20260810 06:49:14.884940 6141 raft_consensus.cc:2454] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d66a9db57b74339813c7648d7a1d423 in term 0.
I20260810 06:49:14.884981 5911 leader_election.cc:302] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 060dcfdb56cd4585aa5bb9bb5e055dff, 4d66a9db57b74339813c7648d7a1d423; no voters:
I20260810 06:49:14.885167 6388 raft_consensus.cc:2781] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:14.885236 6388 raft_consensus.cc:482] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:14.885272 6388 raft_consensus.cc:3037] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:14.886051 6388 raft_consensus.cc:504] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e9824e4c9d4eafa451f8e0df057cf4" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39833 } } peers { permanent_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45207 } } peers { permanent_uuid: "4d66a9db57b74339813c7648d7a1d423" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 39843 } }
I20260810 06:49:14.886252 6388 leader_election.cc:288] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [CANDIDATE]: Term 1 election: Requested vote from peers 70e9824e4c9d4eafa451f8e0df057cf4 (127.15.232.194:39833), 060dcfdb56cd4585aa5bb9bb5e055dff (127.15.232.195:45207)
I20260810 06:49:14.886464 6304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f65e0e806af4666877f19d0e76bc532" candidate_uuid: "4d66a9db57b74339813c7648d7a1d423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff"
I20260810 06:49:14.886525 6141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f65e0e806af4666877f19d0e76bc532" candidate_uuid: "4d66a9db57b74339813c7648d7a1d423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e9824e4c9d4eafa451f8e0df057cf4"
I20260810 06:49:14.886555 6304 raft_consensus.cc:3037] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:14.886631 6141 raft_consensus.cc:3037] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:14.887426 6304 raft_consensus.cc:2454] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d66a9db57b74339813c7648d7a1d423 in term 1.
I20260810 06:49:14.887528 6141 raft_consensus.cc:2454] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d66a9db57b74339813c7648d7a1d423 in term 1.
I20260810 06:49:14.887600 5911 leader_election.cc:302] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 060dcfdb56cd4585aa5bb9bb5e055dff, 4d66a9db57b74339813c7648d7a1d423; no voters:
I20260810 06:49:14.887722 6388 raft_consensus.cc:2781] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:14.887905 6388 raft_consensus.cc:686] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [term 1 LEADER]: Becoming Leader. State: Replica: 4d66a9db57b74339813c7648d7a1d423, State: Running, Role: LEADER
I20260810 06:49:14.888010 6388 consensus_queue.cc:237] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e9824e4c9d4eafa451f8e0df057cf4" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39833 } } peers { permanent_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45207 } } peers { permanent_uuid: "4d66a9db57b74339813c7648d7a1d423" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 39843 } }
I20260810 06:49:14.889006 5818 catalog_manager.cc:5495] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d66a9db57b74339813c7648d7a1d423 (127.15.232.193). New cstate: current_term: 1 leader_uuid: "4d66a9db57b74339813c7648d7a1d423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e9824e4c9d4eafa451f8e0df057cf4" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d66a9db57b74339813c7648d7a1d423" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 39843 } health_report { overall_health: HEALTHY } } }
I20260810 06:49:14.913635 6379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.911065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.15.232.193:39843)
0810 06:49:14.911132 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:14.911135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:14.911144 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:49:14.911189 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:14.911190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:14.911218 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:14.911310 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:14.911469 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.911953 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.912748 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:14.913005 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:14.913094 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:14.913291 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:14.913408 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:14.913503 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:14.913522 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 06:49:14.913544 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:49:14.915676 6304 raft_consensus.cc:1261] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff [term 1 FOLLOWER]: Refusing update from remote peer 4d66a9db57b74339813c7648d7a1d423: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:14.915678 6141 raft_consensus.cc:1261] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4 [term 1 FOLLOWER]: Refusing update from remote peer 4d66a9db57b74339813c7648d7a1d423: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:14.916116 6395 consensus_queue.cc:1035] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "060dcfdb56cd4585aa5bb9bb5e055dff" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 45207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:14.916280 6388 consensus_queue.cc:1035] T 2f65e0e806af4666877f19d0e76bc532 P 4d66a9db57b74339813c7648d7a1d423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70e9824e4c9d4eafa451f8e0df057cf4" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 39833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:14.918922 6399 mvcc.cc:205] Tried to move back new op lower bound from 7316867296931315712 to 7316867296822124544. Current Snapshot: MvccSnapshot[applied={T|T < 7316867296931315712}]
I20260810 06:49:14.919478 6402 mvcc.cc:205] Tried to move back new op lower bound from 7316867296931315712 to 7316867296822124544. Current Snapshot: MvccSnapshot[applied={T|T < 7316867296931315712 or (T in {7316867296931315712})}]
I20260810 06:49:14.922017 5817 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:14.922133 5817 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:14.923014 5817 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d595b556e7e49b2b8c68ee354e0f3ef: Sending DeleteTablet for 3 replicas of tablet 2f65e0e806af4666877f19d0e76bc532
I20260810 06:49:14.923406 5964 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f65e0e806af4666877f19d0e76bc532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:14 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:57378
I20260810 06:49:14.923461 6120 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f65e0e806af4666877f19d0e76bc532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:14 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:57050
I20260810 06:49:14.923728 6278 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f65e0e806af4666877f19d0e76bc532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:14 UTC) from {username='kudu', principal='kudu/127.15.232.254@KRBTEST.COM'} at 127.0.0.1:46694
I20260810 06:49:14.923766 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 5890
I20260810 06:49:14.924135 6412 tablet_replica.cc:330] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4: stopping tablet replica
I20260810 06:49:14.924317 6412 raft_consensus.cc:2229] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:14.924441 6412 raft_consensus.cc:2258] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:14.925486 6412 ts_tablet_manager.cc:1905] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:14.927196 6412 ts_tablet_manager.cc:1918] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:14.927299 6412 log.cc:1196] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/2f65e0e806af4666877f19d0e76bc532
I20260810 06:49:14.927574 6412 ts_tablet_manager.cc:1939] T 2f65e0e806af4666877f19d0e76bc532 P 70e9824e4c9d4eafa451f8e0df057cf4: Deleting consensus metadata
I20260810 06:49:14.928107 5802 catalog_manager.cc:4841] TS 70e9824e4c9d4eafa451f8e0df057cf4 (127.15.232.194:39833): tablet 2f65e0e806af4666877f19d0e76bc532 (table test-table [id=f18138d7b6e542dd84669ad5745e11be]) successfully deleted
I20260810 06:49:14.930115 6413 tablet_replica.cc:330] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff: stopping tablet replica
I20260810 06:49:14.930280 6413 raft_consensus.cc:2229] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:14.930449 6413 raft_consensus.cc:2258] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:14.930971 6413 ts_tablet_manager.cc:1905] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:14.932531 6413 ts_tablet_manager.cc:1918] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:14.932627 6413 log.cc:1196] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/2f65e0e806af4666877f19d0e76bc532
I20260810 06:49:14.932895 6413 ts_tablet_manager.cc:1939] T 2f65e0e806af4666877f19d0e76bc532 P 060dcfdb56cd4585aa5bb9bb5e055dff: Deleting consensus metadata
I20260810 06:49:14.933431 5804 catalog_manager.cc:4841] TS 060dcfdb56cd4585aa5bb9bb5e055dff (127.15.232.195:45207): tablet 2f65e0e806af4666877f19d0e76bc532 (table test-table [id=f18138d7b6e542dd84669ad5745e11be]) successfully deleted
W20260810 06:49:14.934787 5802 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:39843 (error 108)
I20260810 06:49:14.935452 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 6046
I20260810 06:49:14.935473 6378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:14.935297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:39843 (local address 127.0.0.1:57384)
0810 06:49:14.935359 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:14.935428 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:39843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 06:49:14.937744 5802 catalog_manager.cc:4584] TS 4d66a9db57b74339813c7648d7a1d423 (127.15.232.193:39843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2f65e0e806af4666877f19d0e76bc532: Network error: Client connection negotiation failed: client connection to 127.15.232.193:39843: connect: Connection refused (error 111)
I20260810 06:49:14.943413 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 6205
I20260810 06:49:14.951505 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 5782
2026-08-10T06:49:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2349 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T06:49:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:14Z Disabled control of system clock
I20260810 06:49:14.984141 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:46529
--webserver_interface=127.15.232.254
--webserver_port=0
--builtin_ntp_servers=127.15.232.212:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:46529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:49:15.068380 6419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:15.068634 6419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:15.068657 6419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:15.069958 6419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:15.070008 6419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:15.070021 6419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:15.070034 6419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:15.070047 6419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:15.071698 6419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:46529
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:46529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:15.071995 6419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:15.072327 6419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:15.077662 6426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:15.077648 6429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:15.077780 6427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:15.077999 6419 server_base.cc:1034] running on GCE node
I20260810 06:49:15.078372 6419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:15.078724 6419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:15.079921 6419 hybrid_clock.cc:648] HybridClock initialized: now 1786344555079894 us; error 43 us; skew 500 ppm
I20260810 06:49:15.081467 6419 webserver.cc:460] Webserver started at http://127.15.232.254:35181/ using document root <none> and password file <none>
I20260810 06:49:15.081745 6419 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:15.081892 6419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:15.082055 6419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:15.083238 6419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "e0fb0c85a3b24d4bb3d9d75c6b765939"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-vxj2"
server_key: "00f7722382562ac8e310818ac2ccf728"
server_key_iv: "a9e79b54b59b59f40124e6bdb71e0bb5"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.083647 6419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "e0fb0c85a3b24d4bb3d9d75c6b765939"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-vxj2"
server_key: "00f7722382562ac8e310818ac2ccf728"
server_key_iv: "a9e79b54b59b59f40124e6bdb71e0bb5"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.085264 6419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260810 06:49:15.086159 6435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:15.086355 6419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:15.086490 6419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "e0fb0c85a3b24d4bb3d9d75c6b765939"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-vxj2"
server_key: "00f7722382562ac8e310818ac2ccf728"
server_key_iv: "a9e79b54b59b59f40124e6bdb71e0bb5"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.086580 6419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:15.117771 6419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:15.118166 6419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:15.118340 6419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:15.122653 6419 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:46529
I20260810 06:49:15.122694 6497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:46529 every 8 connection(s)
I20260810 06:49:15.123088 6419 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:15.124228 6502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:15.126906 6502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939: Bootstrap starting.
I20260810 06:49:15.127597 6502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:15.127931 6502 log.cc:824] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:15.128875 6502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939: No bootstrap required, opened a new log
I20260810 06:49:15.130142 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 6419
I20260810 06:49:15.130273 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:15.130466 16291 external_mini_cluster.cc:1428] Setting key 2add5809a87c00e2c93aaba0e8e6dd02
I20260810 06:49:15.131312 6502 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fb0c85a3b24d4bb3d9d75c6b765939" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 46529 } }
I20260810 06:49:15.131464 6502 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:15.131487 6502 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0fb0c85a3b24d4bb3d9d75c6b765939, State: Initialized, Role: FOLLOWER
I20260810 06:49:15.131628 6502 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fb0c85a3b24d4bb3d9d75c6b765939" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 46529 } }
I20260810 06:49:15.131703 6502 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:15.131727 6502 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:15.131803 6502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:15.132402 6502 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fb0c85a3b24d4bb3d9d75c6b765939" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 46529 } }
I20260810 06:49:15.132534 6502 leader_election.cc:302] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0fb0c85a3b24d4bb3d9d75c6b765939; no voters:
I20260810 06:49:15.132755 6502 leader_election.cc:288] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:15.132771 6508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:15.132916 6508 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [term 1 LEADER]: Becoming Leader. State: Replica: e0fb0c85a3b24d4bb3d9d75c6b765939, State: Running, Role: LEADER
I20260810 06:49:15.133006 6508 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fb0c85a3b24d4bb3d9d75c6b765939" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 46529 } }
I20260810 06:49:15.133060 6502 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:15.133404 6510 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0fb0c85a3b24d4bb3d9d75c6b765939. Latest consensus state: current_term: 1 leader_uuid: "e0fb0c85a3b24d4bb3d9d75c6b765939" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fb0c85a3b24d4bb3d9d75c6b765939" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 46529 } } }
I20260810 06:49:15.133538 6510 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:15.133404 6509 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0fb0c85a3b24d4bb3d9d75c6b765939" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fb0c85a3b24d4bb3d9d75c6b765939" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 46529 } } }
I20260810 06:49:15.133725 6509 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:15.134490 6514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:15.135150 6514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:15.137117 6514 catalog_manager.cc:1347] Generated new cluster ID: 8f91c6b23d2446bca0d8321d163fdfb7
I20260810 06:49:15.137207 6514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:15.137560 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.131076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40176 (local address 127.15.232.254:46529)
0810 06:49:15.131328 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.131336 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.131389 (+ 53us) server_negotiation.cc:407] Connection header received
0810 06:49:15.131532 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.131549 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.131639 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.131825 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:15.132141 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.133980 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.135225 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.135843 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.136020 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.136807 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:15.136811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:15.136812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:15.137045 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:15.137094 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:15.137108 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:15.137205 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.137299 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.137382 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":125,"threads_started":1}
I20260810 06:49:15.164901 6514 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:15.165617 6514 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:15.175027 6514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939: Generated new TSK 0
I20260810 06:49:15.175334 6514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:15.179958 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.193:0
--local_ip_for_outbound_sockets=127.15.232.193
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:46529
--builtin_ntp_servers=127.15.232.212:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:49:15.292078 6529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:15.292392 6529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:15.292438 6529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:15.293921 6529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:15.293994 6529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:15.294044 6529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.193
I20260810 06:49:15.295658 6529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.193
--webserver_port=0
--tserver_master_addrs=127.15.232.254:46529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.193
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:15.296000 6529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:15.296402 6529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:15.302106 6536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:15.302069 6538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:15.302078 6535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:15.302443 6529 server_base.cc:1034] running on GCE node
I20260810 06:49:15.302650 6529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:15.302922 6529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:15.304100 6529 hybrid_clock.cc:648] HybridClock initialized: now 1786344555304089 us; error 33 us; skew 500 ppm
I20260810 06:49:15.305536 6529 webserver.cc:460] Webserver started at http://127.15.232.193:40605/ using document root <none> and password file <none>
I20260810 06:49:15.305761 6529 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:15.305822 6529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:15.305917 6529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:15.307039 6529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/data/instance:
uuid: "7a8157f43cd7445698cdfb3b914c56c2"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-vxj2"
server_key: "46c8d9c0ea4ba7339283d57b5e1ad560"
server_key_iv: "c5d61dc18c3ca6a600e9bdceb6d75558"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.307356 6529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance:
uuid: "7a8157f43cd7445698cdfb3b914c56c2"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-vxj2"
server_key: "46c8d9c0ea4ba7339283d57b5e1ad560"
server_key_iv: "c5d61dc18c3ca6a600e9bdceb6d75558"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.308692 6529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:15.309666 6545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:15.309908 6529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:49:15.310014 6529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/wal
uuid: "7a8157f43cd7445698cdfb3b914c56c2"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-vxj2"
server_key: "46c8d9c0ea4ba7339283d57b5e1ad560"
server_key_iv: "c5d61dc18c3ca6a600e9bdceb6d75558"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.310077 6529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:15.329404 6529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:15.329746 6529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:15.329877 6529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:15.330096 6529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:15.330504 6529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:15.330547 6529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:15.330574 6529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:15.330638 6529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:15.337604 6529 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.193:38027
I20260810 06:49:15.337684 6681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.193:38027 every 8 connection(s)
I20260810 06:49:15.338088 6529 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/data/info.pb
I20260810 06:49:15.343915 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.338985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:45771 (local address 127.15.232.254:46529)
0810 06:49:15.339067 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.339069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.339355 (+ 286us) server_negotiation.cc:407] Connection header received
0810 06:49:15.339880 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.339881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.339894 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.339958 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:15.340719 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.341386 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.342464 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.342668 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.342738 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.343637 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:15.343638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:15.343639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:15.343644 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:15.343661 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:15.343665 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:15.343717 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.343809 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.343847 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:49:15.344187 6684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.339004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:46529 (local address 127.15.232.193:45771)
0810 06:49:15.339200 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:15.339236 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:15.339758 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:15.340045 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:15.340049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:15.340058 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:15.340608 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.340613 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.341488 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.341489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.342352 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.342354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.342757 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.342758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.342908 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.343488 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:49:15.343489 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:15.343521 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:49:15.343546 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:15.343727 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:15.343732 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:15.343734 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:49:15.343789 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 06:49:15.343912 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260810 06:49:15.344848 6682 heartbeater.cc:347] Connected to a master server at 127.15.232.254:46529
I20260810 06:49:15.344975 6682 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:15.345219 6682 heartbeater.cc:511] Master 127.15.232.254:46529 requested a full tablet report, sending...
I20260810 06:49:15.345826 6455 ts_manager.cc:194] Registered new tserver with Master: 7a8157f43cd7445698cdfb3b914c56c2 (127.15.232.193:38027)
I20260810 06:49:15.346472 6455 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.193:45771
I20260810 06:49:15.347170 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 6529
I20260810 06:49:15.347324 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-0/wal/instance
I20260810 06:49:15.347541 16291 external_mini_cluster.cc:1428] Setting key 6ce2f3eac0618d19b8a9ff517430ff4a
I20260810 06:49:15.348631 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.194:0
--local_ip_for_outbound_sockets=127.15.232.194
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:46529
--builtin_ntp_servers=127.15.232.212:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:49:15.438736 6685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:15.439011 6685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:15.439050 6685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:15.440220 6685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:15.440289 6685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:15.440330 6685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.194
I20260810 06:49:15.441736 6685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.194
--webserver_port=0
--tserver_master_addrs=127.15.232.254:46529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.194
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:15.442046 6685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:15.442384 6685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:15.447927 6693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:15.447921 6692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:15.447997 6685 server_base.cc:1034] running on GCE node
W20260810 06:49:15.447947 6696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:15.448359 6685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:15.448647 6685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:15.449823 6685 hybrid_clock.cc:648] HybridClock initialized: now 1786344555449794 us; error 40 us; skew 500 ppm
I20260810 06:49:15.451297 6685 webserver.cc:460] Webserver started at http://127.15.232.194:35323/ using document root <none> and password file <none>
I20260810 06:49:15.451603 6685 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:15.451685 6685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:15.451849 6685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:15.453054 6685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/data/instance:
uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-vxj2"
server_key: "ecc3f4ab339f704bb95ee3c89e8e079a"
server_key_iv: "67a1249b08660f68fca81de9b09ceb57"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.453516 6685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance:
uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-vxj2"
server_key: "ecc3f4ab339f704bb95ee3c89e8e079a"
server_key_iv: "67a1249b08660f68fca81de9b09ceb57"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.455092 6685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:49:15.455878 6703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:15.456058 6685 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:49:15.456171 6685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/wal
uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-vxj2"
server_key: "ecc3f4ab339f704bb95ee3c89e8e079a"
server_key_iv: "67a1249b08660f68fca81de9b09ceb57"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.456254 6685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:15.473263 6685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:15.473644 6685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:15.473814 6685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:15.474140 6685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:15.474514 6685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:15.474570 6685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:15.474618 6685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:15.474640 6685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:15.481230 6685 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.194:44525
I20260810 06:49:15.481333 6836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.194:44525 every 8 connection(s)
I20260810 06:49:15.481758 6685 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/data/info.pb
I20260810 06:49:15.484609 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 6685
I20260810 06:49:15.484768 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/wal/instance
I20260810 06:49:15.484977 16291 external_mini_cluster.cc:1428] Setting key c6e9de8119b55a619374c9e2b4a42db0
I20260810 06:49:15.486054 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.195:0
--local_ip_for_outbound_sockets=127.15.232.195
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:46529
--builtin_ntp_servers=127.15.232.212:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:49:15.488189 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.482700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.194:49471 (local address 127.15.232.254:46529)
0810 06:49:15.482754 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.482756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.483032 (+ 276us) server_negotiation.cc:407] Connection header received
0810 06:49:15.483587 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.483588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.483600 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.483667 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:15.484362 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.484993 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.486526 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.486756 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.486847 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.487876 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:15.487877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:15.487877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:15.487883 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:15.487901 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:15.487906 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:15.487960 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.488068 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.488106 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:49:15.488452 6839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.482673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:46529 (local address 127.15.232.194:49471)
0810 06:49:15.482900 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:15.482939 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:15.483449 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:15.483724 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:15.483728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:15.483737 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:15.484268 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.484273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.485124 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.485126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.486394 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.486399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.486865 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.486866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.487034 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.487683 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:49:15.487684 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:15.487718 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:49:15.487753 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:15.487969 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:15.487975 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:15.487978 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:49:15.488071 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 06:49:15.488194 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260810 06:49:15.489074 6837 heartbeater.cc:347] Connected to a master server at 127.15.232.254:46529
I20260810 06:49:15.489218 6837 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:15.489447 6837 heartbeater.cc:511] Master 127.15.232.254:46529 requested a full tablet report, sending...
I20260810 06:49:15.489936 6455 ts_manager.cc:194] Registered new tserver with Master: a9ff8eb9d8a74049a9e1b3ab922986ca (127.15.232.194:44525)
I20260810 06:49:15.490453 6455 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.194:49471
W20260810 06:49:15.573774 6841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:15.573998 6841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:15.574021 6841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:15.575250 6841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:15.575301 6841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:15.575330 6841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.195
I20260810 06:49:15.576829 6841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.195
--webserver_port=0
--tserver_master_addrs=127.15.232.254:46529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.195
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:15.577124 6841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:15.577492 6841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:15.583189 6847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:15.583184 6848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:15.583292 6841 server_base.cc:1034] running on GCE node
W20260810 06:49:15.583184 6850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:15.583667 6841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:15.583902 6841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:15.585059 6841 hybrid_clock.cc:648] HybridClock initialized: now 1786344555585049 us; error 32 us; skew 500 ppm
I20260810 06:49:15.586624 6841 webserver.cc:460] Webserver started at http://127.15.232.195:35831/ using document root <none> and password file <none>
I20260810 06:49:15.586853 6841 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:15.586901 6841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:15.587016 6841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:15.588165 6841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/data/instance:
uuid: "167cd7c226004af2aceca82d3f1c62e8"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-vxj2"
server_key: "b41ba3b3b9b3e81535a68657699b1fda"
server_key_iv: "109ddb3b400bc80fba784708c5b9b721"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.588500 6841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance:
uuid: "167cd7c226004af2aceca82d3f1c62e8"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-vxj2"
server_key: "b41ba3b3b9b3e81535a68657699b1fda"
server_key_iv: "109ddb3b400bc80fba784708c5b9b721"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.590106 6841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:15.591009 6857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:15.591216 6841 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:15.591353 6841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/wal
uuid: "167cd7c226004af2aceca82d3f1c62e8"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-vxj2"
server_key: "b41ba3b3b9b3e81535a68657699b1fda"
server_key_iv: "109ddb3b400bc80fba784708c5b9b721"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.591452 6841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:15.612434 6841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:15.612847 6841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:15.613032 6841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:15.613366 6841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:15.613804 6841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:15.613875 6841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:15.613920 6841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:15.613947 6841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:15.620152 6841 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.195:46533
I20260810 06:49:15.620276 6997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.195:46533 every 8 connection(s)
I20260810 06:49:15.620626 6841 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/data/info.pb
I20260810 06:49:15.622587 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 6841
I20260810 06:49:15.622707 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/wal/instance
I20260810 06:49:15.622918 16291 external_mini_cluster.cc:1428] Setting key 9e3189999399c23f1f8cac7d43b135f0
I20260810 06:49:15.626642 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.621528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.195:43521 (local address 127.15.232.254:46529)
0810 06:49:15.621582 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.621583 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.621859 (+ 276us) server_negotiation.cc:407] Connection header received
0810 06:49:15.622426 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.622427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.622438 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.622492 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:15.623545 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.624194 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.625309 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.625502 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.625562 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.626406 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:15.626407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:15.626407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:15.626412 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:15.626427 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:15.626431 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:15.626488 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.626551 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.626579 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:49:15.626842 7000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.621516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:46529 (local address 127.15.232.195:43521)
0810 06:49:15.621710 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:15.621745 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:15.622296 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:15.622582 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:15.622588 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:15.622599 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:15.623427 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.623434 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.624285 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.624286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.625217 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.625219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.625583 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.625584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.625698 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.626267 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:49:15.626267 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:15.626299 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:49:15.626322 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:15.626488 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:15.626491 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:15.626493 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:49:15.626542 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 06:49:15.626624 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260810 06:49:15.627422 6998 heartbeater.cc:347] Connected to a master server at 127.15.232.254:46529
I20260810 06:49:15.627519 6998 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:15.627676 6998 heartbeater.cc:511] Master 127.15.232.254:46529 requested a full tablet report, sending...
I20260810 06:49:15.628083 6455 ts_manager.cc:194] Registered new tserver with Master: 167cd7c226004af2aceca82d3f1c62e8 (127.15.232.195:46533)
I20260810 06:49:15.628543 6455 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.195:43521
I20260810 06:49:15.635078 16291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:49:15.639618 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.636677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40190 (local address 127.15.232.254:46529)
0810 06:49:15.636746 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.636748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.636899 (+ 151us) server_negotiation.cc:407] Connection header received
0810 06:49:15.636943 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.636944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.636955 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.637005 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:15.637122 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.637683 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.638447 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.638628 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.638680 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.639379 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:15.639380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:15.639381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:15.639385 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:15.639399 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:15.639402 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:15.639452 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.639529 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.639554 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:49:15.641515 6455 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:49:15.643204 6455 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:15.650332 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.646058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35258 (local address 127.15.232.193:38027)
0810 06:49:15.646242 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.646248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.646260 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:49:15.646317 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.646319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.646334 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.646430 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:15.646633 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.647428 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.648736 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.649814 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.649901 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.650046 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.650189 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.650235 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260810 06:49:15.650331 7009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.645782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:38027 (local address 127.0.0.1:35258)
0810 06:49:15.646181 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:15.646193 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:15.646242 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:15.646448 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:15.646451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:15.646464 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:15.646546 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.646549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.647524 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.647526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.648631 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.648634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.649932 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.649933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.650047 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.650066 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 06:49:15.650162 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:49:15.650225 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":72,"threads_started":1}
I20260810 06:49:15.651011 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.645772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:46533 (local address 127.0.0.1:37354)
0810 06:49:15.645912 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:15.645959 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:15.646035 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:15.646687 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:15.646688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:15.646690 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:15.646756 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.646758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.648156 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.648157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.649412 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.649416 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.650404 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.650776 (+ 372us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.650846 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.650852 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:49:15.650894 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 06:49:15.650940 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":91,"threads_started":1}
I20260810 06:49:15.651259 6604 tablet_service.cc:1467] Processing CreateTablet for tablet 2a0594d512f34490b56b08444cd623ee (DEFAULT_TABLE table=test-table [id=18c2f19e385a450894234ab768ba1647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:15.651619 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.645981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37354 (local address 127.15.232.195:46533)
0810 06:49:15.646278 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.646283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.646292 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:49:15.646333 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.646335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.646350 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.646835 (+ 485us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:15.646914 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.648073 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.649557 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.650299 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.651340 (+ 1041us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.651455 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.651486 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.651536 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":111,"threads_started":1}
I20260810 06:49:15.651746 6604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a0594d512f34490b56b08444cd623ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:15.652153 7010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.645782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:44525 (local address 127.0.0.1:43704)
0810 06:49:15.646528 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:15.646540 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:15.646595 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:15.646821 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:15.646822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:15.646824 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:15.646855 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.646857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.649556 (+ 2699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.649557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.650660 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.650664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.651877 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.651878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.651949 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.651953 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:49:15.652032 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:49:15.652079 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260810 06:49:15.652179 7014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.646057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43704 (local address 127.15.232.194:44525)
0810 06:49:15.646269 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.646274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.646595 (+ 321us) server_negotiation.cc:407] Connection header received
0810 06:49:15.646662 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.646665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.646680 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.646751 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:15.646931 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.648057 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.650780 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.651598 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.651702 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.651797 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.652051 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.652093 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":104,"threads_started":1}
I20260810 06:49:15.652282 6913 tablet_service.cc:1467] Processing CreateTablet for tablet 2a0594d512f34490b56b08444cd623ee (DEFAULT_TABLE table=test-table [id=18c2f19e385a450894234ab768ba1647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:15.652635 6913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a0594d512f34490b56b08444cd623ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:15.653879 6757 tablet_service.cc:1467] Processing CreateTablet for tablet 2a0594d512f34490b56b08444cd623ee (DEFAULT_TABLE table=test-table [id=18c2f19e385a450894234ab768ba1647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:15.654268 6757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a0594d512f34490b56b08444cd623ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:15.655663 7016 tablet_bootstrap.cc:492] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2: Bootstrap starting.
I20260810 06:49:15.656276 7017 tablet_bootstrap.cc:492] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8: Bootstrap starting.
I20260810 06:49:15.656518 7016 tablet_bootstrap.cc:654] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:15.656858 7016 log.cc:824] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:15.656970 7017 tablet_bootstrap.cc:654] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:15.657343 7017 log.cc:824] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:15.658017 7016 tablet_bootstrap.cc:492] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2: No bootstrap required, opened a new log
I20260810 06:49:15.658077 7016 ts_tablet_manager.cc:1397] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:49:15.658177 7017 tablet_bootstrap.cc:492] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8: No bootstrap required, opened a new log
I20260810 06:49:15.658273 7017 ts_tablet_manager.cc:1397] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 06:49:15.660112 7016 raft_consensus.cc:348] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167cd7c226004af2aceca82d3f1c62e8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 46533 } } peers { permanent_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44525 } } peers { permanent_uuid: "7a8157f43cd7445698cdfb3b914c56c2" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38027 } }
I20260810 06:49:15.660255 7016 raft_consensus.cc:374] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:15.660279 7016 raft_consensus.cc:729] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a8157f43cd7445698cdfb3b914c56c2, State: Initialized, Role: FOLLOWER
I20260810 06:49:15.660211 7017 raft_consensus.cc:348] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167cd7c226004af2aceca82d3f1c62e8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 46533 } } peers { permanent_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44525 } } peers { permanent_uuid: "7a8157f43cd7445698cdfb3b914c56c2" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38027 } }
I20260810 06:49:15.660398 7017 raft_consensus.cc:374] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:15.660441 7017 raft_consensus.cc:729] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 167cd7c226004af2aceca82d3f1c62e8, State: Initialized, Role: FOLLOWER
I20260810 06:49:15.660413 7016 consensus_queue.cc:260] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167cd7c226004af2aceca82d3f1c62e8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 46533 } } peers { permanent_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44525 } } peers { permanent_uuid: "7a8157f43cd7445698cdfb3b914c56c2" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38027 } }
I20260810 06:49:15.660552 7017 consensus_queue.cc:260] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167cd7c226004af2aceca82d3f1c62e8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 46533 } } peers { permanent_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44525 } } peers { permanent_uuid: "7a8157f43cd7445698cdfb3b914c56c2" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38027 } }
I20260810 06:49:15.660733 6682 heartbeater.cc:503] Master 127.15.232.254:46529 was elected leader, sending a full tablet report...
I20260810 06:49:15.660715 7016 ts_tablet_manager.cc:1428] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:49:15.660856 7017 ts_tablet_manager.cc:1428] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:49:15.660997 6998 heartbeater.cc:503] Master 127.15.232.254:46529 was elected leader, sending a full tablet report...
I20260810 06:49:15.660880 7020 tablet_bootstrap.cc:492] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca: Bootstrap starting.
I20260810 06:49:15.662276 7020 tablet_bootstrap.cc:654] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:15.662613 7020 log.cc:824] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:15.663358 7020 tablet_bootstrap.cc:492] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca: No bootstrap required, opened a new log
I20260810 06:49:15.663419 7020 ts_tablet_manager.cc:1397] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 06:49:15.665433 7020 raft_consensus.cc:348] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167cd7c226004af2aceca82d3f1c62e8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 46533 } } peers { permanent_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44525 } } peers { permanent_uuid: "7a8157f43cd7445698cdfb3b914c56c2" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38027 } }
I20260810 06:49:15.665606 7020 raft_consensus.cc:374] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:15.665634 7020 raft_consensus.cc:729] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ff8eb9d8a74049a9e1b3ab922986ca, State: Initialized, Role: FOLLOWER
I20260810 06:49:15.665755 7020 consensus_queue.cc:260] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167cd7c226004af2aceca82d3f1c62e8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 46533 } } peers { permanent_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44525 } } peers { permanent_uuid: "7a8157f43cd7445698cdfb3b914c56c2" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38027 } }
I20260810 06:49:15.666064 7020 ts_tablet_manager.cc:1428] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:49:15.666102 6837 heartbeater.cc:503] Master 127.15.232.254:46529 was elected leader, sending a full tablet report...
I20260810 06:49:15.707837 7022 raft_consensus.cc:482] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:15.707988 7022 raft_consensus.cc:504] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167cd7c226004af2aceca82d3f1c62e8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 46533 } } peers { permanent_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44525 } } peers { permanent_uuid: "7a8157f43cd7445698cdfb3b914c56c2" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38027 } }
I20260810 06:49:15.708371 7022 leader_election.cc:288] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 167cd7c226004af2aceca82d3f1c62e8 (127.15.232.195:46533), a9ff8eb9d8a74049a9e1b3ab922986ca (127.15.232.194:44525)
I20260810 06:49:15.712791 6684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.708548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.195:46533 (local address 127.15.232.193:51541)
0810 06:49:15.708602 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:15.708616 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:15.708670 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:15.708921 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:15.708922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:15.708923 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:15.709079 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.709081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.710131 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.710131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.711531 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.711536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.712455 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.712456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.712541 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.712548 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 06:49:15.712614 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 06:49:15.712668 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":98}
I20260810 06:49:15.712821 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.708618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:51541 (local address 127.15.232.195:46533)
0810 06:49:15.708679 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.708682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.708697 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:49:15.708733 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.708734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.708812 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.708910 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:15.709210 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.710038 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.711671 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.712316 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.712420 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.712430 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.712643 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.712694 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 06:49:15.712809 7014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.708695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.193:58957 (local address 127.15.232.194:44525)
0810 06:49:15.708737 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.708739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.708850 (+ 111us) server_negotiation.cc:407] Connection header received
0810 06:49:15.708882 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.708883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.708897 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.708957 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:15.709212 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.709988 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.711676 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.712316 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.712420 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.712430 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.712660 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.712708 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:49:15.712877 7027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.708590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.194:44525 (local address 127.15.232.193:58957)
0810 06:49:15.708763 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:15.708771 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:15.708827 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:15.708976 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:15.708977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:15.708978 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:15.709052 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.709054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.710120 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.710122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.711531 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:15.711536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.712475 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.712475 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:15.712541 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.712548 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 06:49:15.712613 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 06:49:15.712668 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":85,"thread_start_us":107,"threads_started":1}
I20260810 06:49:15.713220 6938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0594d512f34490b56b08444cd623ee" candidate_uuid: "7a8157f43cd7445698cdfb3b914c56c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "167cd7c226004af2aceca82d3f1c62e8" is_pre_election: true
I20260810 06:49:15.713272 6781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0594d512f34490b56b08444cd623ee" candidate_uuid: "7a8157f43cd7445698cdfb3b914c56c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca" is_pre_election: true
I20260810 06:49:15.713411 6938 raft_consensus.cc:2454] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a8157f43cd7445698cdfb3b914c56c2 in term 0.
I20260810 06:49:15.713481 6781 raft_consensus.cc:2454] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a8157f43cd7445698cdfb3b914c56c2 in term 0.
I20260810 06:49:15.713732 6548 leader_election.cc:302] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 167cd7c226004af2aceca82d3f1c62e8, 7a8157f43cd7445698cdfb3b914c56c2; no voters:
I20260810 06:49:15.713923 7022 raft_consensus.cc:2781] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:15.714011 7022 raft_consensus.cc:482] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:15.714063 7022 raft_consensus.cc:3037] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:15.714942 7022 raft_consensus.cc:504] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167cd7c226004af2aceca82d3f1c62e8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 46533 } } peers { permanent_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44525 } } peers { permanent_uuid: "7a8157f43cd7445698cdfb3b914c56c2" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38027 } }
I20260810 06:49:15.715140 7022 leader_election.cc:288] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [CANDIDATE]: Term 1 election: Requested vote from peers 167cd7c226004af2aceca82d3f1c62e8 (127.15.232.195:46533), a9ff8eb9d8a74049a9e1b3ab922986ca (127.15.232.194:44525)
I20260810 06:49:15.715333 6781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0594d512f34490b56b08444cd623ee" candidate_uuid: "7a8157f43cd7445698cdfb3b914c56c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca"
I20260810 06:49:15.715386 6938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0594d512f34490b56b08444cd623ee" candidate_uuid: "7a8157f43cd7445698cdfb3b914c56c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "167cd7c226004af2aceca82d3f1c62e8"
I20260810 06:49:15.715435 6781 raft_consensus.cc:3037] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:15.715523 6938 raft_consensus.cc:3037] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:15.716320 6938 raft_consensus.cc:2454] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a8157f43cd7445698cdfb3b914c56c2 in term 1.
I20260810 06:49:15.716339 6781 raft_consensus.cc:2454] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a8157f43cd7445698cdfb3b914c56c2 in term 1.
I20260810 06:49:15.716565 6547 leader_election.cc:302] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a8157f43cd7445698cdfb3b914c56c2, a9ff8eb9d8a74049a9e1b3ab922986ca; no voters:
I20260810 06:49:15.716804 7022 raft_consensus.cc:2781] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:15.717074 7022 raft_consensus.cc:686] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [term 1 LEADER]: Becoming Leader. State: Replica: 7a8157f43cd7445698cdfb3b914c56c2, State: Running, Role: LEADER
I20260810 06:49:15.717222 7022 consensus_queue.cc:237] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167cd7c226004af2aceca82d3f1c62e8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 46533 } } peers { permanent_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44525 } } peers { permanent_uuid: "7a8157f43cd7445698cdfb3b914c56c2" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38027 } }
I20260810 06:49:15.718231 6455 catalog_manager.cc:5495] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a8157f43cd7445698cdfb3b914c56c2 (127.15.232.193). New cstate: current_term: 1 leader_uuid: "7a8157f43cd7445698cdfb3b914c56c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167cd7c226004af2aceca82d3f1c62e8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 46533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a8157f43cd7445698cdfb3b914c56c2" member_type: VOTER last_known_addr { host: "127.15.232.193" port: 38027 } health_report { overall_health: HEALTHY } } }
I20260810 06:49:15.725014 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.722240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35262 (local address 127.15.232.193:38027)
0810 06:49:15.722350 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.722352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.722362 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:49:15.722394 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.722395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.722413 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.722477 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:15.722641 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.723101 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.724014 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:15.724223 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:15.724306 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:15.724494 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:15.724611 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:15.724683 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.724743 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.724774 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 06:49:15.727291 6938 raft_consensus.cc:1261] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8 [term 1 FOLLOWER]: Refusing update from remote peer 7a8157f43cd7445698cdfb3b914c56c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:15.727329 6781 raft_consensus.cc:1261] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca [term 1 FOLLOWER]: Refusing update from remote peer 7a8157f43cd7445698cdfb3b914c56c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:15.727762 7028 consensus_queue.cc:1035] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "167cd7c226004af2aceca82d3f1c62e8" member_type: VOTER last_known_addr { host: "127.15.232.195" port: 46533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:15.727919 7022 consensus_queue.cc:1035] T 2a0594d512f34490b56b08444cd623ee P 7a8157f43cd7445698cdfb3b914c56c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ff8eb9d8a74049a9e1b3ab922986ca" member_type: VOTER last_known_addr { host: "127.15.232.194" port: 44525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:15.731320 7035 mvcc.cc:205] Tried to move back new op lower bound from 7316867300255834112 to 7316867300218523648. Current Snapshot: MvccSnapshot[applied={T|T < 7316867300255834112}]
I20260810 06:49:15.732833 7034 mvcc.cc:205] Tried to move back new op lower bound from 7316867300255834112 to 7316867300218523648. Current Snapshot: MvccSnapshot[applied={T|T < 7316867300255834112 or (T in {7316867300255834112})}]
I20260810 06:49:15.733263 6455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:15.733393 6455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:15.734252 6455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0fb0c85a3b24d4bb3d9d75c6b765939: Sending DeleteTablet for 3 replicas of tablet 2a0594d512f34490b56b08444cd623ee
I20260810 06:49:15.734694 6757 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a0594d512f34490b56b08444cd623ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:15 UTC) from {username='slave'} at 127.0.0.1:43704
I20260810 06:49:15.734701 6913 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a0594d512f34490b56b08444cd623ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:15 UTC) from {username='slave'} at 127.0.0.1:37354
I20260810 06:49:15.735052 7043 tablet_replica.cc:330] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8: stopping tablet replica
I20260810 06:49:15.735174 7043 raft_consensus.cc:2229] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:15.735322 7043 raft_consensus.cc:2258] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:15.735502 7042 tablet_replica.cc:330] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca: stopping tablet replica
I20260810 06:49:15.735462 6604 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a0594d512f34490b56b08444cd623ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:15 UTC) from {username='slave'} at 127.0.0.1:35258
I20260810 06:49:15.735507 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 6529
I20260810 06:49:15.735616 7042 raft_consensus.cc:2229] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:15.735720 7042 raft_consensus.cc:2258] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:15.738998 7042 ts_tablet_manager.cc:1905] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:15.739698 7043 ts_tablet_manager.cc:1905] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:15.741681 7042 ts_tablet_manager.cc:1918] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:15.741761 7042 log.cc:1196] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-1/wal/wals/2a0594d512f34490b56b08444cd623ee
I20260810 06:49:15.741832 7043 ts_tablet_manager.cc:1918] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:15.741933 7043 log.cc:1196] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8: Deleting WAL directory at /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344303981681-16291-0/minicluster-data/ts-2/wal/wals/2a0594d512f34490b56b08444cd623ee
I20260810 06:49:15.742013 7042 ts_tablet_manager.cc:1939] T 2a0594d512f34490b56b08444cd623ee P a9ff8eb9d8a74049a9e1b3ab922986ca: Deleting consensus metadata
I20260810 06:49:15.742238 7043 ts_tablet_manager.cc:1939] T 2a0594d512f34490b56b08444cd623ee P 167cd7c226004af2aceca82d3f1c62e8: Deleting consensus metadata
I20260810 06:49:15.742730 6436 catalog_manager.cc:4841] TS a9ff8eb9d8a74049a9e1b3ab922986ca (127.15.232.194:44525): tablet 2a0594d512f34490b56b08444cd623ee (table test-table [id=18c2f19e385a450894234ab768ba1647]) successfully deleted
I20260810 06:49:15.743757 6437 catalog_manager.cc:4841] TS 167cd7c226004af2aceca82d3f1c62e8 (127.15.232.195:46533): tablet 2a0594d512f34490b56b08444cd623ee (table test-table [id=18c2f19e385a450894234ab768ba1647]) successfully deleted
W20260810 06:49:15.747378 6439 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.193:38027 (error 108)
I20260810 06:49:15.747974 7010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.747807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.193:38027 (local address 127.0.0.1:35268)
0810 06:49:15.747880 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:15.747943 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.193:38027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 06:49:15.748083 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 6685
W20260810 06:49:15.748148 6439 catalog_manager.cc:4584] TS 7a8157f43cd7445698cdfb3b914c56c2 (127.15.232.193:38027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2a0594d512f34490b56b08444cd623ee: Network error: Client connection negotiation failed: client connection to 127.15.232.193:38027: connect: Connection refused (error 111)
I20260810 06:49:15.759531 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 6841
I20260810 06:49:15.767299 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 6419
2026-08-10T06:49:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (813 ms)
[----------] 6 tests from EncryptionPolicyTest (9400 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:15 dist-test-slave-vxj2 krb5kdc[7049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:15 dist-test-slave-vxj2 krb5kdc[7049](info): set up 2 sockets
Aug 10 06:49:15 dist-test-slave-vxj2 krb5kdc[7049](info): commencing operation
krb5kdc: starting...
W20260810 06:49:16.900233 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:16 dist-test-slave-vxj2 krb5kdc[7049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:17.032833 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40351
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:17.134624 7065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:17.134845 7065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:17.134877 7065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:17.136081 7065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:17.136132 7065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:17.136145 7065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:17.136158 7065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:17.136170 7065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:17.137876 7065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40351
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:17.138160 7065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:17.138419 7065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:17.143994 7073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:17.144166 7065 server_base.cc:1034] running on GCE node
W20260810 06:49:17.143968 7071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:17.143970 7075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:17.144522 7065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:17.144852 7065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:17.146041 7065 hybrid_clock.cc:648] HybridClock initialized: now 1786344557146029 us; error 47 us; skew 500 ppm
Aug 10 06:49:17 dist-test-slave-vxj2 krb5kdc[7049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344557, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:17.148612 7065 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:17.149230 7065 webserver.cc:460] Webserver started at http://127.0.0.1:33821/ using document root <none> and password file <none>
I20260810 06:49:17.149495 7065 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:17.149569 7065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:17.149725 7065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:17.151427 7065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "a439dc15de934ace813202c0a78a1ab7"
format_stamp: "Formatted at 2026-08-10 06:49:17 on dist-test-slave-vxj2"
server_key: "1da708f26bad2f9b33ed5f2e18e3e22d"
server_key_iv: "f69d02874c6c004ff576c9000125ea54"
server_key_version: "encryptionkey@0"
I20260810 06:49:17.151893 7065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "a439dc15de934ace813202c0a78a1ab7"
format_stamp: "Formatted at 2026-08-10 06:49:17 on dist-test-slave-vxj2"
server_key: "1da708f26bad2f9b33ed5f2e18e3e22d"
server_key_iv: "f69d02874c6c004ff576c9000125ea54"
server_key_version: "encryptionkey@0"
I20260810 06:49:17.153641 7065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260810 06:49:17.154564 7086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:17.154774 7065 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:17.154911 7065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "a439dc15de934ace813202c0a78a1ab7"
format_stamp: "Formatted at 2026-08-10 06:49:17 on dist-test-slave-vxj2"
server_key: "1da708f26bad2f9b33ed5f2e18e3e22d"
server_key_iv: "f69d02874c6c004ff576c9000125ea54"
server_key_version: "encryptionkey@0"
I20260810 06:49:17.155009 7065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:17.187157 7065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:17.189666 7065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:17.189842 7065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:17.194047 7065 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40351
I20260810 06:49:17.194089 7147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40351 every 8 connection(s)
I20260810 06:49:17.194451 7065 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:17.195356 7148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:17.198109 7148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7: Bootstrap starting.
I20260810 06:49:17.198776 7148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:17.199088 7148 log.cc:824] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:17.199496 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 7065
I20260810 06:49:17.199642 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:17.199862 16291 external_mini_cluster.cc:1428] Setting key 378d22d8418705b119c7750432c9c807
I20260810 06:49:17.200007 7148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7: No bootstrap required, opened a new log
I20260810 06:49:17.201989 7148 raft_consensus.cc:348] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439dc15de934ace813202c0a78a1ab7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40351 } }
I20260810 06:49:17.202168 7148 raft_consensus.cc:374] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:17.202216 7148 raft_consensus.cc:729] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a439dc15de934ace813202c0a78a1ab7, State: Initialized, Role: FOLLOWER
I20260810 06:49:17.202376 7148 consensus_queue.cc:260] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439dc15de934ace813202c0a78a1ab7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40351 } }
I20260810 06:49:17.202471 7148 raft_consensus.cc:388] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:17.202517 7148 raft_consensus.cc:482] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:17.202593 7148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:17.203599 7148 raft_consensus.cc:504] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439dc15de934ace813202c0a78a1ab7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40351 } }
I20260810 06:49:17.203751 7148 leader_election.cc:302] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a439dc15de934ace813202c0a78a1ab7; no voters:
I20260810 06:49:17.203964 7148 leader_election.cc:288] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:17.203995 7155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:17.204114 7155 raft_consensus.cc:686] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [term 1 LEADER]: Becoming Leader. State: Replica: a439dc15de934ace813202c0a78a1ab7, State: Running, Role: LEADER
I20260810 06:49:17.204209 7155 consensus_queue.cc:237] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439dc15de934ace813202c0a78a1ab7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40351 } }
I20260810 06:49:17.204269 7148 sys_catalog.cc:564] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:17.204624 7157 sys_catalog.cc:455] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a439dc15de934ace813202c0a78a1ab7. Latest consensus state: current_term: 1 leader_uuid: "a439dc15de934ace813202c0a78a1ab7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439dc15de934ace813202c0a78a1ab7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40351 } } }
I20260810 06:49:17.204725 7157 sys_catalog.cc:458] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:17.204885 7156 sys_catalog.cc:455] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a439dc15de934ace813202c0a78a1ab7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a439dc15de934ace813202c0a78a1ab7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40351 } } }
I20260810 06:49:17.204936 7156 sys_catalog.cc:458] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:17.205041 7160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:17.205593 7160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:17.207525 7160 catalog_manager.cc:1347] Generated new cluster ID: 5bc36305c11a45e3b516a9beaf41f954
I20260810 06:49:17.207587 7160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:49:17 dist-test-slave-vxj2 krb5kdc[7049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:49:17.213023 7154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:17.200501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48259 (local address 127.0.0.1:40351)
0810 06:49:17.200694 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:17.200705 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:17.200740 (+ 35us) server_negotiation.cc:407] Connection header received
0810 06:49:17.201970 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:17.201980 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:17.202043 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:17.202216 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:17.203035 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:17.204129 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:17.205281 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:17.205984 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:17.206108 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:17.209186 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:17.209191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:17.209191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:17.209205 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:17.211178 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:17.211668 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:17.211670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:17.211671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:17.211811 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:17.212075 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:17.212076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:17.212076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:17.212386 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:17.212536 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:17.212684 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 06:49:17.212793 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260810 06:49:17.214149 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:17.225214 7154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:17.217503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39377 (local address 127.0.0.1:40351)
0810 06:49:17.217539 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:17.217542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:17.217634 (+ 92us) server_negotiation.cc:407] Connection header received
0810 06:49:17.217695 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:17.217696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:17.217705 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:17.217769 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:17.218459 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:17.219194 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:17.220228 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:17.220552 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:17.220611 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:17.222149 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:17.222151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:17.222151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:17.222159 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:17.223873 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:17.224310 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:17.224312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:17.224312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:17.224368 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:17.224603 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:17.224604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:17.224604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:17.224730 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:17.224839 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:17.224994 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 06:49:17.225061 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 06:49:17.238711 7160 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:17.239344 7160 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:17.244899 7160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a439dc15de934ace813202c0a78a1ab7: Generated new TSK 0
I20260810 06:49:17.245221 7160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:17.251842 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 7065
W20260810 06:49:17.258174 7047 connection.cc:535] client connection to 127.0.0.1:40351 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40351): Connection reset by peer (error 104)
2026-08-10T06:49:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1482 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:17 dist-test-slave-vxj2 krb5kdc[7189](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:17 dist-test-slave-vxj2 krb5kdc[7189](info): set up 2 sockets
Aug 10 06:49:17 dist-test-slave-vxj2 krb5kdc[7189](info): commencing operation
krb5kdc: starting...
W20260810 06:49:18.386140 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:18 dist-test-slave-vxj2 krb5kdc[7189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:18.512874 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:18.599169 7208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:18.599416 7208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:18.599443 7208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:18.600692 7208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:18.600746 7208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:18.600761 7208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:18.600775 7208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:18.600821 7208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:18.602444 7208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44319
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:18.602736 7208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:18.602994 7208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:18.608321 7213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:18.608289 7217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:18.608496 7208 server_base.cc:1034] running on GCE node
W20260810 06:49:18.608590 7214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:18.608858 7208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:18.609139 7208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:18.610363 7208 hybrid_clock.cc:648] HybridClock initialized: now 1786344558610339 us; error 40 us; skew 500 ppm
Aug 10 06:49:18 dist-test-slave-vxj2 krb5kdc[7189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344558, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:18.613065 7208 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:18.613576 7208 webserver.cc:460] Webserver started at http://127.0.0.1:34829/ using document root <none> and password file <none>
I20260810 06:49:18.613790 7208 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:18.613838 7208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:18.613943 7208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:18.615022 7208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "459abf586e32436a8599befcbd71dfc5"
format_stamp: "Formatted at 2026-08-10 06:49:18 on dist-test-slave-vxj2"
server_key: "7409f5058e06932ab8fa589b02b73a13"
server_key_iv: "d5003cb8e598d2de91e026b12625e1a6"
server_key_version: "encryptionkey@0"
I20260810 06:49:18.615343 7208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "459abf586e32436a8599befcbd71dfc5"
format_stamp: "Formatted at 2026-08-10 06:49:18 on dist-test-slave-vxj2"
server_key: "7409f5058e06932ab8fa589b02b73a13"
server_key_iv: "d5003cb8e598d2de91e026b12625e1a6"
server_key_version: "encryptionkey@0"
I20260810 06:49:18.616708 7208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:49:18.617570 7226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:18.617754 7208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:49:18.617918 7208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "459abf586e32436a8599befcbd71dfc5"
format_stamp: "Formatted at 2026-08-10 06:49:18 on dist-test-slave-vxj2"
server_key: "7409f5058e06932ab8fa589b02b73a13"
server_key_iv: "d5003cb8e598d2de91e026b12625e1a6"
server_key_version: "encryptionkey@0"
I20260810 06:49:18.617998 7208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:18.628664 7208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:18.630702 7208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:18.630851 7208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:18.634835 7208 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44319
I20260810 06:49:18.634915 7291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44319 every 8 connection(s)
I20260810 06:49:18.635258 7208 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:18.636242 7292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:18.638252 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 7208
I20260810 06:49:18.638388 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:18.638598 16291 external_mini_cluster.cc:1428] Setting key 5e23df2fa42cb90092d072b1289d1039
I20260810 06:49:18.639025 7292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5: Bootstrap starting.
I20260810 06:49:18.639859 7292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:18.640209 7292 log.cc:824] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:18.641125 7292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5: No bootstrap required, opened a new log
I20260810 06:49:18.643406 7292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459abf586e32436a8599befcbd71dfc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44319 } }
I20260810 06:49:18.643595 7292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:18.643646 7292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 459abf586e32436a8599befcbd71dfc5, State: Initialized, Role: FOLLOWER
I20260810 06:49:18.643803 7292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459abf586e32436a8599befcbd71dfc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44319 } }
I20260810 06:49:18.643908 7292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:18.643934 7292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:18.644032 7292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:18.644768 7292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459abf586e32436a8599befcbd71dfc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44319 } }
I20260810 06:49:18.644894 7292 leader_election.cc:302] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 459abf586e32436a8599befcbd71dfc5; no voters:
I20260810 06:49:18.645079 7292 leader_election.cc:288] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:18.645272 7300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:18.645344 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:18.639185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58747 (local address 127.0.0.1:44319)
0810 06:49:18.639395 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:18.639405 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:18.639443 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:49:18.640085 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:18.640094 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:18.640166 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:18.640331 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:18.640602 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:18.641981 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:18.642901 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:18.643571 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:18.643690 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:18.644543 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:18.644547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:18.644548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:18.644764 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:18.644807 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:18.644823 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:18.644918 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:18.645053 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 06:49:18.645085 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260810 06:49:18.645371 7292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:18.645454 7300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [term 1 LEADER]: Becoming Leader. State: Replica: 459abf586e32436a8599befcbd71dfc5, State: Running, Role: LEADER
I20260810 06:49:18.645851 7300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459abf586e32436a8599befcbd71dfc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44319 } }
I20260810 06:49:18.646237 7302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 459abf586e32436a8599befcbd71dfc5. Latest consensus state: current_term: 1 leader_uuid: "459abf586e32436a8599befcbd71dfc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459abf586e32436a8599befcbd71dfc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44319 } } }
I20260810 06:49:18.646330 7302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:18.646587 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:18.646636 7301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "459abf586e32436a8599befcbd71dfc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459abf586e32436a8599befcbd71dfc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44319 } } }
I20260810 06:49:18.646730 7301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:18.646895 7314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:18.647413 7314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:18.649221 7314 catalog_manager.cc:1347] Generated new cluster ID: a4698da283af4a1cb36de5169c0fbcca
I20260810 06:49:18.649323 7314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:18.651314 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:18.648249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54983 (local address 127.0.0.1:44319)
0810 06:49:18.648285 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:18.648287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:18.648445 (+ 158us) server_negotiation.cc:407] Connection header received
0810 06:49:18.648477 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:18.648478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:18.648489 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:18.648561 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:18.648698 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:18.649291 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:18.650048 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:18.650281 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:18.650378 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:18.651067 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:18.651069 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:18.651069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:18.651074 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:18.651089 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:18.651093 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:18.651146 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:18.651207 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 06:49:18.651216 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:49:18.672628 7314 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:18.673460 7314 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:18.680311 7314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 459abf586e32436a8599befcbd71dfc5: Generated new TSK 0
I20260810 06:49:18.680565 7314 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:18.687062 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 7208
W20260810 06:49:18.694814 7187 connection.cc:535] client connection to 127.0.0.1:44319 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44319): Connection reset by peer (error 104)
2026-08-10T06:49:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1436 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:18 dist-test-slave-vxj2 krb5kdc[7332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:18 dist-test-slave-vxj2 krb5kdc[7332](info): set up 2 sockets
Aug 10 06:49:18 dist-test-slave-vxj2 krb5kdc[7332](info): commencing operation
krb5kdc: starting...
W20260810 06:49:19.811340 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:19 dist-test-slave-vxj2 krb5kdc[7332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:19.942651 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:20.027755 7350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:20.028028 7350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:20.028067 7350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:20.029244 7350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:20.029326 7350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:20.029350 7350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:20.029371 7350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:20.029397 7350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:20.030946 7350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42293
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:20.031243 7350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:20.031486 7350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:20.036789 7355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:20.036809 7356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:20.036980 7350 server_base.cc:1034] running on GCE node
W20260810 06:49:20.036789 7358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:20.037398 7350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:20.037703 7350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:20.038872 7350 hybrid_clock.cc:648] HybridClock initialized: now 1786344560038847 us; error 39 us; skew 500 ppm
Aug 10 06:49:20 dist-test-slave-vxj2 krb5kdc[7332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344560, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:20.041872 7350 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:20.042361 7350 webserver.cc:460] Webserver started at http://127.0.0.1:33367/ using document root <none> and password file <none>
I20260810 06:49:20.042626 7350 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:20.042755 7350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:20.042928 7350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:20.044081 7350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "4d8d4a7917f44d16a972bbcc1a1e2b5c"
format_stamp: "Formatted at 2026-08-10 06:49:20 on dist-test-slave-vxj2"
server_key: "9ad1fadf59473f48b2251a05b0fe8e34"
server_key_iv: "c5958fd5a6ea4f9d352633127ddcbb92"
server_key_version: "encryptionkey@0"
I20260810 06:49:20.044476 7350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "4d8d4a7917f44d16a972bbcc1a1e2b5c"
format_stamp: "Formatted at 2026-08-10 06:49:20 on dist-test-slave-vxj2"
server_key: "9ad1fadf59473f48b2251a05b0fe8e34"
server_key_iv: "c5958fd5a6ea4f9d352633127ddcbb92"
server_key_version: "encryptionkey@0"
I20260810 06:49:20.046082 7350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:20.047021 7366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:20.047214 7350 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:20.047322 7350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "4d8d4a7917f44d16a972bbcc1a1e2b5c"
format_stamp: "Formatted at 2026-08-10 06:49:20 on dist-test-slave-vxj2"
server_key: "9ad1fadf59473f48b2251a05b0fe8e34"
server_key_iv: "c5958fd5a6ea4f9d352633127ddcbb92"
server_key_version: "encryptionkey@0"
I20260810 06:49:20.047417 7350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:20.051119 7350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:20.053622 7350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:20.053798 7350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:20.058126 7350 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42293
I20260810 06:49:20.058197 7429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42293 every 8 connection(s)
I20260810 06:49:20.058581 7350 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:20.059525 7430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:20.062076 7430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c: Bootstrap starting.
I20260810 06:49:20.062721 7430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:20.063028 7430 log.cc:824] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:20.063782 7430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c: No bootstrap required, opened a new log
I20260810 06:49:20.065577 7430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8d4a7917f44d16a972bbcc1a1e2b5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260810 06:49:20.065726 7430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:20.065749 7430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8d4a7917f44d16a972bbcc1a1e2b5c, State: Initialized, Role: FOLLOWER
I20260810 06:49:20.065922 7430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8d4a7917f44d16a972bbcc1a1e2b5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260810 06:49:20.065991 7430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:20.066013 7430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:20.066085 7430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:20.066679 7430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8d4a7917f44d16a972bbcc1a1e2b5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260810 06:49:20.066772 7430 leader_election.cc:302] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d8d4a7917f44d16a972bbcc1a1e2b5c; no voters:
I20260810 06:49:20.066979 7430 leader_election.cc:288] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:20.067071 7433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:20.067315 7430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:20.067327 7433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [term 1 LEADER]: Becoming Leader. State: Replica: 4d8d4a7917f44d16a972bbcc1a1e2b5c, State: Running, Role: LEADER
I20260810 06:49:20.067529 7433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8d4a7917f44d16a972bbcc1a1e2b5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260810 06:49:20.068076 7434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d8d4a7917f44d16a972bbcc1a1e2b5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8d4a7917f44d16a972bbcc1a1e2b5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } } }
I20260810 06:49:20.068172 7434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:20.068235 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 7350
I20260810 06:49:20.068380 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:20.068518 7442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:20.068555 7435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d8d4a7917f44d16a972bbcc1a1e2b5c. Latest consensus state: current_term: 1 leader_uuid: "4d8d4a7917f44d16a972bbcc1a1e2b5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8d4a7917f44d16a972bbcc1a1e2b5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } } }
I20260810 06:49:20.068617 16291 external_mini_cluster.cc:1428] Setting key b0fbd0f5736d1562980f302f9ad4a41e
I20260810 06:49:20.068639 7435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:20.069129 7442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:20.071180 7442 catalog_manager.cc:1347] Generated new cluster ID: 92a8428c73bc444a91b2aa4ff3bfa70d
I20260810 06:49:20.071244 7442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:20.071470 7453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:20.069392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47955 (local address 127.0.0.1:42293)
0810 06:49:20.069572 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:20.069582 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:20.069616 (+ 34us) server_negotiation.cc:407] Connection header received
0810 06:49:20.069743 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:20.069751 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:20.069786 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:20.069932 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:20.070851 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:20.070854 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:20.070855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:20.070860 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:20.070917 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:20.070934 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:20.071225 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:20.071256 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 06:49:20.071298 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260810 06:49:20.072467 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:20.075362 7453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:20.074189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41917 (local address 127.0.0.1:42293)
0810 06:49:20.074227 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:20.074229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:20.074241 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:49:20.074315 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:20.074316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:20.074331 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:20.074385 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:20.075032 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:20.075034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:20.075034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:20.075039 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:20.075054 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:20.075059 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:20.075115 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:20.075196 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 06:49:20.075209 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:49:20.076392 7442 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:20.076552 7442 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:20.086385 7442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d8d4a7917f44d16a972bbcc1a1e2b5c: Generated new TSK 0
I20260810 06:49:20.086627 7442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:20.092427 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 7350
2026-08-10T06:49:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:20 dist-test-slave-vxj2 krb5kdc[7465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:20 dist-test-slave-vxj2 krb5kdc[7465](info): set up 2 sockets
Aug 10 06:49:20 dist-test-slave-vxj2 krb5kdc[7465](info): commencing operation
krb5kdc: starting...
W20260810 06:49:21.213322 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:21 dist-test-slave-vxj2 krb5kdc[7465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:21.340260 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:21.430244 7484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:21.430492 7484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:21.430518 7484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:21.431663 7484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:21.431700 7484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:21.431715 7484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:21.431727 7484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:21.431740 7484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:21.433205 7484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:21.433415 7484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:21.433599 7484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:21.438509 7491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:21.438617 7484 server_base.cc:1034] running on GCE node
W20260810 06:49:21.438509 7493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:21.438573 7490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:21.439047 7484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:21.439361 7484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:21.440515 7484 hybrid_clock.cc:648] HybridClock initialized: now 1786344561440493 us; error 39 us; skew 500 ppm
Aug 10 06:49:21 dist-test-slave-vxj2 krb5kdc[7465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344561, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:21.443040 7484 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:21.443548 7484 webserver.cc:460] Webserver started at http://127.0.0.1:46111/ using document root <none> and password file <none>
I20260810 06:49:21.443835 7484 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:21.443908 7484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:21.444064 7484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:21.445191 7484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "56b4a63147324eab82543d9bb16758e7"
format_stamp: "Formatted at 2026-08-10 06:49:21 on dist-test-slave-vxj2"
server_key: "2bbf3b116e53561966e9c3c914c76232"
server_key_iv: "4a84a416d519e0685caa4bda7e9e3e30"
server_key_version: "encryptionkey@0"
I20260810 06:49:21.445591 7484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "56b4a63147324eab82543d9bb16758e7"
format_stamp: "Formatted at 2026-08-10 06:49:21 on dist-test-slave-vxj2"
server_key: "2bbf3b116e53561966e9c3c914c76232"
server_key_iv: "4a84a416d519e0685caa4bda7e9e3e30"
server_key_version: "encryptionkey@0"
I20260810 06:49:21.447055 7484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:49:21.447943 7501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:21.448148 7484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:21.448267 7484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "56b4a63147324eab82543d9bb16758e7"
format_stamp: "Formatted at 2026-08-10 06:49:21 on dist-test-slave-vxj2"
server_key: "2bbf3b116e53561966e9c3c914c76232"
server_key_iv: "4a84a416d519e0685caa4bda7e9e3e30"
server_key_version: "encryptionkey@0"
I20260810 06:49:21.448400 7484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:21.466986 7484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:21.469130 7484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:21.469354 7484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:21.473676 7484 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40701
I20260810 06:49:21.473716 7563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40701 every 8 connection(s)
I20260810 06:49:21.474120 7484 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:21.475080 7564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:21.475777 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 7484
I20260810 06:49:21.475898 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:21.476087 16291 external_mini_cluster.cc:1428] Setting key 0195113b44797c334cc3e9e33eed4818
I20260810 06:49:21.478479 7564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7: Bootstrap starting.
I20260810 06:49:21.479103 7564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:21.479417 7564 log.cc:824] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:21.480221 7564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7: No bootstrap required, opened a new log
I20260810 06:49:21.482920 7564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b4a63147324eab82543d9bb16758e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40701 } }
I20260810 06:49:21.483073 7564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:21.483095 7564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b4a63147324eab82543d9bb16758e7, State: Initialized, Role: FOLLOWER
I20260810 06:49:21.483232 7564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b4a63147324eab82543d9bb16758e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40701 } }
I20260810 06:49:21.483304 7564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:21.483327 7564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:21.483392 7564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [term 0 FOLLOWER]: Advancing to term 1
Aug 10 06:49:21 dist-test-slave-vxj2 krb5kdc[7465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:49:21.484074 7564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b4a63147324eab82543d9bb16758e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40701 } }
I20260810 06:49:21.484192 7564 leader_election.cc:302] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56b4a63147324eab82543d9bb16758e7; no voters:
I20260810 06:49:21.484392 7564 leader_election.cc:288] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:21.484432 7571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:21.484555 7571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [term 1 LEADER]: Becoming Leader. State: Replica: 56b4a63147324eab82543d9bb16758e7, State: Running, Role: LEADER
I20260810 06:49:21.484629 7571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b4a63147324eab82543d9bb16758e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40701 } }
I20260810 06:49:21.484685 7564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:21.485232 7573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56b4a63147324eab82543d9bb16758e7. Latest consensus state: current_term: 1 leader_uuid: "56b4a63147324eab82543d9bb16758e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b4a63147324eab82543d9bb16758e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40701 } } }
I20260810 06:49:21.485625 7573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:21.486421 7582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:21.485510 7572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56b4a63147324eab82543d9bb16758e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b4a63147324eab82543d9bb16758e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40701 } } }
I20260810 06:49:21.486907 7572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:21.487183 7582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:21.488683 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:21.476659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32883 (local address 127.0.0.1:40701)
0810 06:49:21.476839 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:21.476848 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:21.476874 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:49:21.477361 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:21.477368 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:21.477426 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:21.477544 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:21.478387 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:21.479452 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:21.480565 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:21.480975 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:21.481065 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:21.484123 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:21.484129 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:21.484129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:21.484144 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:21.486772 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:21.487271 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:21.487275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:21.487275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:21.487383 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:21.487626 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:21.487627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:21.487627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:21.487980 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:21.488143 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:21.488287 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 06:49:21.488446 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260810 06:49:21.489257 7582 catalog_manager.cc:1347] Generated new cluster ID: 4b9ee0247cd1424781ec4c7f62530cb4
I20260810 06:49:21.489312 7582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:21.493813 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:21.502601 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:21.495375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47031 (local address 127.0.0.1:40701)
0810 06:49:21.495421 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:21.495423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:21.495580 (+ 157us) server_negotiation.cc:407] Connection header received
0810 06:49:21.495670 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:21.495671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:21.495683 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:21.495746 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:21.496376 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:21.496878 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:21.497846 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:21.498112 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:21.498217 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:21.499565 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:21.499566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:21.499567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:21.499574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:21.501287 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:21.501696 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:21.501697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:21.501697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:21.501767 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:21.502016 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:21.502017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:21.502018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:21.502147 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:21.502275 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:21.502401 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 06:49:21.502483 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:49:21.524312 7582 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:21.524914 7582 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:21.538892 7582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56b4a63147324eab82543d9bb16758e7: Generated new TSK 0
I20260810 06:49:21.539155 7582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:21.542691 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 7484
2026-08-10T06:49:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:21 dist-test-slave-vxj2 krb5kdc[7601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:21 dist-test-slave-vxj2 krb5kdc[7601](info): set up 2 sockets
Aug 10 06:49:21 dist-test-slave-vxj2 krb5kdc[7601](info): commencing operation
krb5kdc: starting...
W20260810 06:49:22.669677 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:22 dist-test-slave-vxj2 krb5kdc[7601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:22.799803 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:22.884078 7619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:22.884331 7619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:22.884374 7619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:22.885574 7619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:22.885615 7619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:22.885630 7619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:22.885649 7619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:22.885661 7619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:22.887115 7619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:22.887316 7619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:22.887487 7619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:22.892413 7629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:22.892501 7619 server_base.cc:1034] running on GCE node
W20260810 06:49:22.892374 7631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:22.892349 7628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:22.892886 7619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:22.893262 7619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:22.894472 7619 hybrid_clock.cc:648] HybridClock initialized: now 1786344562894456 us; error 39 us; skew 500 ppm
Aug 10 06:49:22 dist-test-slave-vxj2 krb5kdc[7601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344562, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:22.902701 7619 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:22.903254 7619 webserver.cc:460] Webserver started at http://127.0.0.1:35865/ using document root <none> and password file <none>
I20260810 06:49:22.903515 7619 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:22.903590 7619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:22.903752 7619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:22.904896 7619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "4ed6d3ef620d4d8097a9309ab3788341"
format_stamp: "Formatted at 2026-08-10 06:49:22 on dist-test-slave-vxj2"
server_key: "cbcc59a16aee66db39ce9aa3fc54bafc"
server_key_iv: "81a5c369656ac2c5d0a5ed39ad84a810"
server_key_version: "encryptionkey@0"
I20260810 06:49:22.905333 7619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "4ed6d3ef620d4d8097a9309ab3788341"
format_stamp: "Formatted at 2026-08-10 06:49:22 on dist-test-slave-vxj2"
server_key: "cbcc59a16aee66db39ce9aa3fc54bafc"
server_key_iv: "81a5c369656ac2c5d0a5ed39ad84a810"
server_key_version: "encryptionkey@0"
I20260810 06:49:22.906847 7619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 06:49:22.907683 7642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:22.908000 7619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:22.908120 7619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "4ed6d3ef620d4d8097a9309ab3788341"
format_stamp: "Formatted at 2026-08-10 06:49:22 on dist-test-slave-vxj2"
server_key: "cbcc59a16aee66db39ce9aa3fc54bafc"
server_key_iv: "81a5c369656ac2c5d0a5ed39ad84a810"
server_key_version: "encryptionkey@0"
I20260810 06:49:22.908211 7619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:22.921821 7619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:22.924037 7619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:22.924217 7619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:22.928685 7619 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42585
I20260810 06:49:22.928726 7699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42585 every 8 connection(s)
I20260810 06:49:22.929108 7619 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:22.930091 7701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:22.932479 7701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341: Bootstrap starting.
I20260810 06:49:22.933053 7701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:22.933390 7701 log.cc:824] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:22.934182 7701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341: No bootstrap required, opened a new log
I20260810 06:49:22.935346 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 7619
I20260810 06:49:22.935489 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:22.935684 16291 external_mini_cluster.cc:1428] Setting key e1e6738b40c44cf113e4b089d67e90d6
I20260810 06:49:22.936535 7701 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed6d3ef620d4d8097a9309ab3788341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42585 } }
I20260810 06:49:22.936728 7701 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:22.936774 7701 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed6d3ef620d4d8097a9309ab3788341, State: Initialized, Role: FOLLOWER
I20260810 06:49:22.936955 7701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed6d3ef620d4d8097a9309ab3788341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42585 } }
I20260810 06:49:22.937057 7701 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:22.937081 7701 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:22.937181 7701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:22.937788 7701 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed6d3ef620d4d8097a9309ab3788341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42585 } }
I20260810 06:49:22.937886 7701 leader_election.cc:302] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ed6d3ef620d4d8097a9309ab3788341; no voters:
I20260810 06:49:22.938089 7701 leader_election.cc:288] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:22.938117 7707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:22.938231 7707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [term 1 LEADER]: Becoming Leader. State: Replica: 4ed6d3ef620d4d8097a9309ab3788341, State: Running, Role: LEADER
I20260810 06:49:22.938383 7701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:22.938400 7707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed6d3ef620d4d8097a9309ab3788341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42585 } }
I20260810 06:49:22.940182 7708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ed6d3ef620d4d8097a9309ab3788341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed6d3ef620d4d8097a9309ab3788341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42585 } } }
I20260810 06:49:22.940289 7708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [sys.catalog]: This master's current role is: LEADER
W20260810 06:49:22.940450 7724 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 06:49:22.940515 7709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ed6d3ef620d4d8097a9309ab3788341. Latest consensus state: current_term: 1 leader_uuid: "4ed6d3ef620d4d8097a9309ab3788341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed6d3ef620d4d8097a9309ab3788341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42585 } } }
I20260810 06:49:22.940594 7709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341 [sys.catalog]: This master's current role is: LEADER
W20260810 06:49:22.940510 7724 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:49:22.940711 7726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:22.940930 7726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:22.942783 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:22.936245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45927 (local address 127.0.0.1:42585)
0810 06:49:22.936499 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:22.936509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:22.936545 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:49:22.936663 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:22.936670 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:22.936730 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:22.936884 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:22.937206 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:22.938734 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:22.940037 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:22.940727 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:22.940829 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:22.942056 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:22.942060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:22.942060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:22.942268 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:22.942307 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:22.942322 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:22.942398 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:22.942496 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 06:49:22.942563 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":126,"threads_started":1}
I20260810 06:49:22.942847 7726 catalog_manager.cc:1347] Generated new cluster ID: f03757dc219242b9b454f172ba17569a
I20260810 06:49:22.942895 7726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:22.943754 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:22.948745 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:22.945227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43731 (local address 127.0.0.1:42585)
0810 06:49:22.945271 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:22.945273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:22.945391 (+ 118us) server_negotiation.cc:407] Connection header received
0810 06:49:22.945448 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:22.945449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:22.945463 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:22.945524 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:22.945653 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:22.946354 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:22.947446 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:22.947718 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:22.947787 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:22.948487 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:22.948488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:22.948488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:22.948492 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:22.948507 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:22.948512 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:22.948562 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:22.948646 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 06:49:22.948670 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:49:22.952077 7726 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:22.952612 7726 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:22.955225 7726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ed6d3ef620d4d8097a9309ab3788341: Generated new TSK 0
I20260810 06:49:22.955421 7726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:22.962507 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 7619
2026-08-10T06:49:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:23 dist-test-slave-vxj2 krb5kdc[7737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:23 dist-test-slave-vxj2 krb5kdc[7737](info): set up 2 sockets
Aug 10 06:49:23 dist-test-slave-vxj2 krb5kdc[7737](info): commencing operation
krb5kdc: starting...
W20260810 06:49:24.078575 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:24 dist-test-slave-vxj2 krb5kdc[7737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:24.204756 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46573
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:24.287308 7756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:24.287583 7756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:24.287624 7756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:24.288844 7756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:24.288929 7756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:24.288952 7756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:24.288973 7756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:24.289001 7756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:24.290584 7756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:24.290896 7756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:24.291160 7756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:24.296561 7764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:24.296653 7756 server_base.cc:1034] running on GCE node
W20260810 06:49:24.296653 7762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:24.296533 7766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:24.297336 7756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:24.297676 7756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:24.298895 7756 hybrid_clock.cc:648] HybridClock initialized: now 1786344564298886 us; error 33 us; skew 500 ppm
Aug 10 06:49:24 dist-test-slave-vxj2 krb5kdc[7737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344564, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:24.302008 7756 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:24.302474 7756 webserver.cc:460] Webserver started at http://127.0.0.1:34053/ using document root <none> and password file <none>
I20260810 06:49:24.302697 7756 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:24.302749 7756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:24.302845 7756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:24.303915 7756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "ee546183fb37497494b2685a59c9f9ee"
format_stamp: "Formatted at 2026-08-10 06:49:24 on dist-test-slave-vxj2"
server_key: "1f27849e6b58e6a193d68e451137b2f9"
server_key_iv: "957c652f52097f2ec6eb5198f1421b41"
server_key_version: "encryptionkey@0"
I20260810 06:49:24.304225 7756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "ee546183fb37497494b2685a59c9f9ee"
format_stamp: "Formatted at 2026-08-10 06:49:24 on dist-test-slave-vxj2"
server_key: "1f27849e6b58e6a193d68e451137b2f9"
server_key_iv: "957c652f52097f2ec6eb5198f1421b41"
server_key_version: "encryptionkey@0"
I20260810 06:49:24.305725 7756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:24.306564 7773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:24.306829 7756 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:24.306952 7756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "ee546183fb37497494b2685a59c9f9ee"
format_stamp: "Formatted at 2026-08-10 06:49:24 on dist-test-slave-vxj2"
server_key: "1f27849e6b58e6a193d68e451137b2f9"
server_key_iv: "957c652f52097f2ec6eb5198f1421b41"
server_key_version: "encryptionkey@0"
I20260810 06:49:24.307039 7756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:24.310606 7756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:24.312912 7756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:24.313095 7756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:24.317529 7756 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46573
I20260810 06:49:24.317566 7836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46573 every 8 connection(s)
I20260810 06:49:24.317926 7756 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:24.318780 7837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:24.319947 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 7756
I20260810 06:49:24.320080 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:24.320257 16291 external_mini_cluster.cc:1428] Setting key 350daeb44172cc8bb9fca46f3b1d98d3
I20260810 06:49:24.321642 7837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee: Bootstrap starting.
I20260810 06:49:24.322417 7837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:24.322770 7837 log.cc:824] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:24.322942 7840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.320823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50423 (local address 127.0.0.1:46573)
0810 06:49:24.321017 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.321025 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.321051 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:49:24.321544 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.321550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.321577 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.321698 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.322398 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.322402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.322402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:24.322406 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.322443 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:24.322453 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.322570 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.322648 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.322702 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260810 06:49:24.323536 7837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee: No bootstrap required, opened a new log
I20260810 06:49:24.324004 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:24.325958 7837 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee546183fb37497494b2685a59c9f9ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46573 } }
I20260810 06:49:24.326148 7837 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:24.326198 7837 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee546183fb37497494b2685a59c9f9ee, State: Initialized, Role: FOLLOWER
I20260810 06:49:24.326329 7837 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee546183fb37497494b2685a59c9f9ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46573 } }
I20260810 06:49:24.326448 7837 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:24.326525 7837 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:24.326584 7840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.325433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57211 (local address 127.0.0.1:46573)
0810 06:49:24.325478 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.325480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.325585 (+ 105us) server_negotiation.cc:407] Connection header received
0810 06:49:24.325625 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.325626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.325630 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.325671 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.326214 (+ 543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.326216 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.326216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:24.326221 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.326234 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:24.326238 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.326295 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.326414 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.326429 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:49:24.326587 7837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:24.327332 7837 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee546183fb37497494b2685a59c9f9ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46573 } }
I20260810 06:49:24.327461 7837 leader_election.cc:302] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee546183fb37497494b2685a59c9f9ee; no voters:
I20260810 06:49:24.327626 7837 leader_election.cc:288] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:24.327661 7849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:24.327894 7837 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:24.327922 7849 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [term 1 LEADER]: Becoming Leader. State: Replica: ee546183fb37497494b2685a59c9f9ee, State: Running, Role: LEADER
I20260810 06:49:24.328035 7849 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee546183fb37497494b2685a59c9f9ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46573 } }
I20260810 06:49:24.328423 7850 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee546183fb37497494b2685a59c9f9ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee546183fb37497494b2685a59c9f9ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46573 } } }
I20260810 06:49:24.328440 7852 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee546183fb37497494b2685a59c9f9ee. Latest consensus state: current_term: 1 leader_uuid: "ee546183fb37497494b2685a59c9f9ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee546183fb37497494b2685a59c9f9ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46573 } } }
I20260810 06:49:24.328548 7850 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:24.328583 7852 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:24.328765 7862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:24.329301 7862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:24.330933 7862 catalog_manager.cc:1347] Generated new cluster ID: 3fb9f803c8d8485c94f0b7afe570ed5d
I20260810 06:49:24.330996 7862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:24.346338 7862 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:24.346555 7862 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:24.362836 7862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee546183fb37497494b2685a59c9f9ee: Generated new TSK 0
I20260810 06:49:24.363106 7862 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:24.372697 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 7756
2026-08-10T06:49:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:24 dist-test-slave-vxj2 krb5kdc[7877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:24 dist-test-slave-vxj2 krb5kdc[7877](info): set up 2 sockets
Aug 10 06:49:24 dist-test-slave-vxj2 krb5kdc[7877](info): commencing operation
krb5kdc: starting...
W20260810 06:49:25.520351 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:25 dist-test-slave-vxj2 krb5kdc[7877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:25.648434 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44501
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:25.731293 7892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:25.731514 7892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:25.731539 7892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:25.732635 7892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:25.732677 7892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:25.732692 7892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:25.732704 7892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:25.732717 7892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:25.734170 7892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:25.734385 7892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:25.734568 7892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:25.739619 7899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:25.739672 7900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:25.739868 7892 server_base.cc:1034] running on GCE node
W20260810 06:49:25.739620 7902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:25.740155 7892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:25.740422 7892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:25.741568 7892 hybrid_clock.cc:648] HybridClock initialized: now 1786344565741546 us; error 37 us; skew 500 ppm
Aug 10 06:49:25 dist-test-slave-vxj2 krb5kdc[7877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344565, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:25.744237 7892 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:25.744783 7892 webserver.cc:460] Webserver started at http://127.0.0.1:33609/ using document root <none> and password file <none>
I20260810 06:49:25.745040 7892 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:25.745111 7892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:25.745289 7892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:25.746397 7892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "31ee438691844e1e91d6bf5eeaf11d61"
format_stamp: "Formatted at 2026-08-10 06:49:25 on dist-test-slave-vxj2"
server_key: "115afdd6c6721db20c0c520d0e496296"
server_key_iv: "ea8bb5f65a2473cf859239eaa6752b34"
server_key_version: "encryptionkey@0"
I20260810 06:49:25.746788 7892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "31ee438691844e1e91d6bf5eeaf11d61"
format_stamp: "Formatted at 2026-08-10 06:49:25 on dist-test-slave-vxj2"
server_key: "115afdd6c6721db20c0c520d0e496296"
server_key_iv: "ea8bb5f65a2473cf859239eaa6752b34"
server_key_version: "encryptionkey@0"
I20260810 06:49:25.748198 7892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:25.748978 7914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:25.749197 7892 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:25.749307 7892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "31ee438691844e1e91d6bf5eeaf11d61"
format_stamp: "Formatted at 2026-08-10 06:49:25 on dist-test-slave-vxj2"
server_key: "115afdd6c6721db20c0c520d0e496296"
server_key_iv: "ea8bb5f65a2473cf859239eaa6752b34"
server_key_version: "encryptionkey@0"
I20260810 06:49:25.749402 7892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:25.768664 7892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:25.771143 7892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:25.771320 7892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:25.775601 7892 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44501
I20260810 06:49:25.775640 7977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44501 every 8 connection(s)
I20260810 06:49:25.776046 7892 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:25.776921 7978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:25.779286 7978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61: Bootstrap starting.
I20260810 06:49:25.779886 7978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:25.780174 7978 log.cc:824] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:25.780879 7978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61: No bootstrap required, opened a new log
I20260810 06:49:25.782534 7978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ee438691844e1e91d6bf5eeaf11d61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44501 } }
I20260810 06:49:25.782675 7978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:25.782698 7978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ee438691844e1e91d6bf5eeaf11d61, State: Initialized, Role: FOLLOWER
I20260810 06:49:25.782876 7978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ee438691844e1e91d6bf5eeaf11d61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44501 } }
I20260810 06:49:25.782939 7978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:25.782961 7978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:25.783030 7978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:25.783581 7978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ee438691844e1e91d6bf5eeaf11d61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44501 } }
I20260810 06:49:25.783718 7978 leader_election.cc:302] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31ee438691844e1e91d6bf5eeaf11d61; no voters:
I20260810 06:49:25.783802 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 7892
I20260810 06:49:25.783914 7978 leader_election.cc:288] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:25.783941 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:25.783957 7983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:25.784148 16291 external_mini_cluster.cc:1428] Setting key 3b70d7fcec58379826267827246348bc
I20260810 06:49:25.784145 7983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [term 1 LEADER]: Becoming Leader. State: Replica: 31ee438691844e1e91d6bf5eeaf11d61, State: Running, Role: LEADER
I20260810 06:49:25.784188 7978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:25.784246 7983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ee438691844e1e91d6bf5eeaf11d61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44501 } }
I20260810 06:49:25.784636 7983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31ee438691844e1e91d6bf5eeaf11d61. Latest consensus state: current_term: 1 leader_uuid: "31ee438691844e1e91d6bf5eeaf11d61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ee438691844e1e91d6bf5eeaf11d61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44501 } } }
I20260810 06:49:25.784727 7983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:25.784890 7985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31ee438691844e1e91d6bf5eeaf11d61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ee438691844e1e91d6bf5eeaf11d61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44501 } } }
I20260810 06:49:25.785006 7985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:25.785290 7993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:25.785892 7993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:25.787532 7993 catalog_manager.cc:1347] Generated new cluster ID: dfe0f62b385948d99872d4f1f5b80ee1
I20260810 06:49:25.787588 7993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:49:25 dist-test-slave-vxj2 krb5kdc[7877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:49:25.796536 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:25.784782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:39581 (local address 127.0.0.1:44501)
0810 06:49:25.785002 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.785014 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.785291 (+ 277us) server_negotiation.cc:407] Connection header received
0810 06:49:25.786477 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.786487 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.786553 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.786696 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.787520 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:25.788475 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:25.789358 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:25.789793 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:25.789919 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:25.792399 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:25.792404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:25.792404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:25.792417 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:25.794847 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:25.795268 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:25.795272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:25.795272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:25.795386 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:25.795633 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:25.795634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:25.795635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:25.795901 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:25.796100 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:25.796190 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 06:49:25.796302 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":86,"threads_started":1}
I20260810 06:49:25.797641 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:25.804162 7993 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:25.804863 7993 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:25.805744 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:25.799205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:35277 (local address 127.0.0.1:44501)
0810 06:49:25.799244 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.799246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.799255 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:49:25.799302 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.799303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.799314 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.799369 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.799919 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:25.800425 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:25.801135 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:25.801402 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:25.801483 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:25.802761 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:25.802762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:25.802763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:25.802771 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:25.804400 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:25.804821 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:25.804822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:25.804823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:25.804897 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:25.805197 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:25.805199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:25.805199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:25.805321 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:25.805433 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:25.805597 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 06:49:25.805660 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:49:25.814565 7993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31ee438691844e1e91d6bf5eeaf11d61: Generated new TSK 0
I20260810 06:49:25.814811 7993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:25.819869 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 7892
W20260810 06:49:25.828925 7874 connection.cc:535] client connection to 127.0.0.1:44501 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44501): Connection reset by peer (error 104)
2026-08-10T06:49:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:25 dist-test-slave-vxj2 krb5kdc[8018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:25 dist-test-slave-vxj2 krb5kdc[8018](info): set up 2 sockets
Aug 10 06:49:25 dist-test-slave-vxj2 krb5kdc[8018](info): commencing operation
krb5kdc: starting...
W20260810 06:49:26.952883 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:27 dist-test-slave-vxj2 krb5kdc[8018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:27.081393 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39447
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:27.186580 8033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:27.186830 8033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:27.186856 8033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:27.188167 8033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:27.188216 8033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:27.188232 8033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:27.188246 8033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:27.188258 8033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:27.189828 8033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:27.190047 8033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:27.190230 8033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:27.195416 8043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:27.195416 8042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:27.195451 8045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:27.195734 8033 server_base.cc:1034] running on GCE node
I20260810 06:49:27.196012 8033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:27.196303 8033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:27.197495 8033 hybrid_clock.cc:648] HybridClock initialized: now 1786344567197478 us; error 43 us; skew 500 ppm
Aug 10 06:49:27 dist-test-slave-vxj2 krb5kdc[8018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344567, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:27.200249 8033 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:27.200744 8033 webserver.cc:460] Webserver started at http://127.0.0.1:37037/ using document root <none> and password file <none>
I20260810 06:49:27.200986 8033 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:27.201038 8033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:27.201232 8033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:27.202391 8033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "9504deb1430847a694e3c829bcb9bc01"
format_stamp: "Formatted at 2026-08-10 06:49:27 on dist-test-slave-vxj2"
server_key: "c0e88901c8a59413ef1cd4c5e502515a"
server_key_iv: "27e13328ade478429ddb9150738d7002"
server_key_version: "encryptionkey@0"
I20260810 06:49:27.202790 8033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "9504deb1430847a694e3c829bcb9bc01"
format_stamp: "Formatted at 2026-08-10 06:49:27 on dist-test-slave-vxj2"
server_key: "c0e88901c8a59413ef1cd4c5e502515a"
server_key_iv: "27e13328ade478429ddb9150738d7002"
server_key_version: "encryptionkey@0"
I20260810 06:49:27.204282 8033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:27.205092 8052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:27.205304 8033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:27.205381 8033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "9504deb1430847a694e3c829bcb9bc01"
format_stamp: "Formatted at 2026-08-10 06:49:27 on dist-test-slave-vxj2"
server_key: "c0e88901c8a59413ef1cd4c5e502515a"
server_key_iv: "27e13328ade478429ddb9150738d7002"
server_key_version: "encryptionkey@0"
I20260810 06:49:27.205435 8033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:27.220122 8033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:27.222426 8033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:27.222541 8033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:27.226446 8033 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39447
I20260810 06:49:27.226518 8115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39447 every 8 connection(s)
I20260810 06:49:27.226876 8033 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:27.227842 8120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:27.227824 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 8033
I20260810 06:49:27.227957 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:27.228181 16291 external_mini_cluster.cc:1428] Setting key eac2a32be28fbe39c536feefcf287b70
I20260810 06:49:27.231246 8120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01: Bootstrap starting.
I20260810 06:49:27.231959 8120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:27.232247 8120 log.cc:824] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:27.232977 8120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01: No bootstrap required, opened a new log
I20260810 06:49:27.234139 8125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:27.228740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:49773 (local address 127.0.0.1:39447)
0810 06:49:27.228923 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:27.228931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:27.228957 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:49:27.229525 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:27.229533 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:27.229585 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:27.229703 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:27.230069 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:27.230956 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:27.232063 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:27.232537 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:27.232656 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:27.233459 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:27.233463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:27.233464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:27.233646 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:27.233675 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:27.233686 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:27.233775 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:27.233947 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 06:49:27.233978 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260810 06:49:27.235185 8120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9504deb1430847a694e3c829bcb9bc01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39447 } }
I20260810 06:49:27.235386 8120 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:27.235402 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:27.235438 8120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9504deb1430847a694e3c829bcb9bc01, State: Initialized, Role: FOLLOWER
I20260810 06:49:27.235563 8120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9504deb1430847a694e3c829bcb9bc01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39447 } }
I20260810 06:49:27.235641 8120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:27.235684 8120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:27.235733 8120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:27.236303 8120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9504deb1430847a694e3c829bcb9bc01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39447 } }
I20260810 06:49:27.236426 8120 leader_election.cc:302] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9504deb1430847a694e3c829bcb9bc01; no voters:
I20260810 06:49:27.236634 8120 leader_election.cc:288] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:27.236668 8132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:27.236860 8132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [term 1 LEADER]: Becoming Leader. State: Replica: 9504deb1430847a694e3c829bcb9bc01, State: Running, Role: LEADER
I20260810 06:49:27.236919 8120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:27.236951 8132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9504deb1430847a694e3c829bcb9bc01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39447 } }
I20260810 06:49:27.237308 8133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9504deb1430847a694e3c829bcb9bc01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9504deb1430847a694e3c829bcb9bc01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39447 } } }
I20260810 06:49:27.237422 8133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:27.237318 8135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9504deb1430847a694e3c829bcb9bc01. Latest consensus state: current_term: 1 leader_uuid: "9504deb1430847a694e3c829bcb9bc01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9504deb1430847a694e3c829bcb9bc01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39447 } } }
I20260810 06:49:27.237502 8135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01 [sys.catalog]: This master's current role is: LEADER
W20260810 06:49:27.238896 8152 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:49:27.238960 8152 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:49:27.239024 8142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:27.239203 8142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:27.240934 8142 catalog_manager.cc:1347] Generated new cluster ID: 172f48f084124d8d828bd90f5944e826
I20260810 06:49:27.240998 8142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:27.241127 8125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:27.236772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:60571 (local address 127.0.0.1:39447)
0810 06:49:27.236809 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:27.236811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:27.237436 (+ 625us) server_negotiation.cc:407] Connection header received
0810 06:49:27.237688 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:27.237689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:27.237702 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:27.237775 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:27.237912 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:27.238689 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:27.239609 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:27.239830 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:27.239914 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:27.240723 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:27.240724 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:27.240724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:27.240729 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:27.240745 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:27.240749 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:27.240820 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:27.241034 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 06:49:27.241049 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:49:27.253093 8142 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:27.253867 8142 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:27.258278 8142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9504deb1430847a694e3c829bcb9bc01: Generated new TSK 0
I20260810 06:49:27.258471 8142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:27.262001 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 8033
W20260810 06:49:27.269094 8016 connection.cc:535] client connection to 127.0.0.1:39447 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39447): Connection reset by peer (error 104)
2026-08-10T06:49:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:27 dist-test-slave-vxj2 krb5kdc[8159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:27 dist-test-slave-vxj2 krb5kdc[8159](info): set up 2 sockets
Aug 10 06:49:27 dist-test-slave-vxj2 krb5kdc[8159](info): commencing operation
krb5kdc: starting...
W20260810 06:49:28.386965 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:28 dist-test-slave-vxj2 krb5kdc[8159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:28.520267 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41325
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:28.611930 8173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:28.612185 8173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:28.612210 8173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:28.613471 8173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:28.613521 8173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:28.613536 8173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:28.613587 8173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:28.613612 8173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:28.615223 8173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:28.615511 8173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:28.615715 8173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:28.620885 8184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:28.620869 8182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:28.620855 8181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:28.620966 8173 server_base.cc:1034] running on GCE node
I20260810 06:49:28.621385 8173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:28.621717 8173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:28.622884 8173 hybrid_clock.cc:648] HybridClock initialized: now 1786344568622860 us; error 51 us; skew 500 ppm
Aug 10 06:49:28 dist-test-slave-vxj2 krb5kdc[8159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344568, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:28.629657 8173 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:28.630182 8173 webserver.cc:460] Webserver started at http://127.0.0.1:45069/ using document root <none> and password file <none>
I20260810 06:49:28.630432 8173 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:28.630501 8173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:28.630648 8173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:28.631786 8173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "f1437a14f0484bad8013cf250b4b8070"
format_stamp: "Formatted at 2026-08-10 06:49:28 on dist-test-slave-vxj2"
server_key: "ee8d98a209764557bd53bee7a10e57d3"
server_key_iv: "5640c73495d4830ffc025d52983fe7cb"
server_key_version: "encryptionkey@0"
I20260810 06:49:28.632166 8173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "f1437a14f0484bad8013cf250b4b8070"
format_stamp: "Formatted at 2026-08-10 06:49:28 on dist-test-slave-vxj2"
server_key: "ee8d98a209764557bd53bee7a10e57d3"
server_key_iv: "5640c73495d4830ffc025d52983fe7cb"
server_key_version: "encryptionkey@0"
I20260810 06:49:28.633623 8173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:28.634410 8192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:28.634615 8173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:49:28.634753 8173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "f1437a14f0484bad8013cf250b4b8070"
format_stamp: "Formatted at 2026-08-10 06:49:28 on dist-test-slave-vxj2"
server_key: "ee8d98a209764557bd53bee7a10e57d3"
server_key_iv: "5640c73495d4830ffc025d52983fe7cb"
server_key_version: "encryptionkey@0"
I20260810 06:49:28.634832 8173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:28.638466 8173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:28.640578 8173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:28.640749 8173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:28.645040 8173 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41325
I20260810 06:49:28.645074 8249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41325 every 8 connection(s)
I20260810 06:49:28.645493 8173 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:28.645924 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 8173
I20260810 06:49:28.646054 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:28.646260 16291 external_mini_cluster.cc:1428] Setting key c4a7b288235c6f7d977994cd8b247df9
I20260810 06:49:28.646648 8250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:28.649293 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:28.646862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:45615 (local address 127.0.0.1:41325)
0810 06:49:28.647082 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:28.647092 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:28.647125 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:49:28.647789 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:28.647800 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:28.647835 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:28.647999 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:28.648709 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:28.648713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:28.648714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:28.648719 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:28.648764 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:28.648778 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:28.648868 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:28.649013 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 06:49:28.649059 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":101,"threads_started":1}
I20260810 06:49:28.650245 8250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070: Bootstrap starting.
I20260810 06:49:28.650311 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:28.651024 8250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:28.651382 8250 log.cc:824] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:28.652212 8250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070: No bootstrap required, opened a new log
I20260810 06:49:28.652977 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:28.652008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:33285 (local address 127.0.0.1:41325)
0810 06:49:28.652050 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:28.652052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:28.652127 (+ 75us) server_negotiation.cc:407] Connection header received
0810 06:49:28.652202 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:28.652202 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:28.652206 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:28.652250 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:28.652763 (+ 513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:28.652764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:28.652764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:28.652768 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:28.652780 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:28.652783 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:28.652822 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:28.652885 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 06:49:28.652895 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:49:28.654533 8250 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1437a14f0484bad8013cf250b4b8070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41325 } }
I20260810 06:49:28.654716 8250 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:28.654783 8250 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1437a14f0484bad8013cf250b4b8070, State: Initialized, Role: FOLLOWER
I20260810 06:49:28.654924 8250 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1437a14f0484bad8013cf250b4b8070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41325 } }
I20260810 06:49:28.655033 8250 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:28.655092 8250 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:28.655143 8250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:28.655701 8250 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1437a14f0484bad8013cf250b4b8070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41325 } }
I20260810 06:49:28.655815 8250 leader_election.cc:302] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1437a14f0484bad8013cf250b4b8070; no voters:
I20260810 06:49:28.656018 8250 leader_election.cc:288] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:28.656069 8260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:28.656278 8260 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [term 1 LEADER]: Becoming Leader. State: Replica: f1437a14f0484bad8013cf250b4b8070, State: Running, Role: LEADER
I20260810 06:49:28.656280 8250 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:28.656435 8260 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1437a14f0484bad8013cf250b4b8070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41325 } }
I20260810 06:49:28.656823 8261 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1437a14f0484bad8013cf250b4b8070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1437a14f0484bad8013cf250b4b8070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41325 } } }
I20260810 06:49:28.656833 8264 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1437a14f0484bad8013cf250b4b8070. Latest consensus state: current_term: 1 leader_uuid: "f1437a14f0484bad8013cf250b4b8070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1437a14f0484bad8013cf250b4b8070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41325 } } }
I20260810 06:49:28.656935 8261 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:28.656960 8264 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:28.657145 8273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:28.657697 8273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:28.659231 8273 catalog_manager.cc:1347] Generated new cluster ID: 4336d9480afa4fd1be58215d6f280a8c
I20260810 06:49:28.659282 8273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:28.681783 8273 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:28.681968 8273 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:28.686583 8273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1437a14f0484bad8013cf250b4b8070: Generated new TSK 0
I20260810 06:49:28.686803 8273 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:28.691398 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 8173
2026-08-10T06:49:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:28 dist-test-slave-vxj2 krb5kdc[8292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:28 dist-test-slave-vxj2 krb5kdc[8292](info): set up 2 sockets
Aug 10 06:49:28 dist-test-slave-vxj2 krb5kdc[8292](info): commencing operation
krb5kdc: starting...
W20260810 06:49:29.814214 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:29 dist-test-slave-vxj2 krb5kdc[8292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:29.945380 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46753
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:30.029699 8307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:30.029973 8307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:30.030016 8307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:30.031204 8307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:30.031275 8307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:30.031296 8307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:30.031317 8307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:30.031347 8307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:30.032861 8307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46753
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:30.033169 8307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:30.033401 8307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:30.038623 8313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:30.038589 8316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:30.038679 8307 server_base.cc:1034] running on GCE node
W20260810 06:49:30.038589 8314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:30.039062 8307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:30.039393 8307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:30.040584 8307 hybrid_clock.cc:648] HybridClock initialized: now 1786344570040574 us; error 38 us; skew 500 ppm
Aug 10 06:49:30 dist-test-slave-vxj2 krb5kdc[8292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344570, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:30.043444 8307 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:30.043937 8307 webserver.cc:460] Webserver started at http://127.0.0.1:45695/ using document root <none> and password file <none>
I20260810 06:49:30.044194 8307 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:30.044270 8307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:30.044418 8307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:30.045585 8307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "c6ba45b3bed043b5bc5943b63d0e3f7a"
format_stamp: "Formatted at 2026-08-10 06:49:30 on dist-test-slave-vxj2"
server_key: "2852a401a7f7c3b873c2a62abba941ef"
server_key_iv: "094ab442a4fc5bfbd7e06edff7442524"
server_key_version: "encryptionkey@0"
I20260810 06:49:30.045958 8307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "c6ba45b3bed043b5bc5943b63d0e3f7a"
format_stamp: "Formatted at 2026-08-10 06:49:30 on dist-test-slave-vxj2"
server_key: "2852a401a7f7c3b873c2a62abba941ef"
server_key_iv: "094ab442a4fc5bfbd7e06edff7442524"
server_key_version: "encryptionkey@0"
I20260810 06:49:30.047407 8307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:49:30.048169 8326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:30.048348 8307 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:49:30.048466 8307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "c6ba45b3bed043b5bc5943b63d0e3f7a"
format_stamp: "Formatted at 2026-08-10 06:49:30 on dist-test-slave-vxj2"
server_key: "2852a401a7f7c3b873c2a62abba941ef"
server_key_iv: "094ab442a4fc5bfbd7e06edff7442524"
server_key_version: "encryptionkey@0"
I20260810 06:49:30.048540 8307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:30.064033 8307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:30.066166 8307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:30.066339 8307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:30.070538 8307 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46753
I20260810 06:49:30.070582 8386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46753 every 8 connection(s)
I20260810 06:49:30.070995 8307 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:30.071893 8387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:30.074338 8387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a: Bootstrap starting.
I20260810 06:49:30.074888 8387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:30.075155 8387 log.cc:824] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:30.075832 8387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a: No bootstrap required, opened a new log
I20260810 06:49:30.077476 8387 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ba45b3bed043b5bc5943b63d0e3f7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46753 } }
I20260810 06:49:30.077620 8387 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:30.077642 8387 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6ba45b3bed043b5bc5943b63d0e3f7a, State: Initialized, Role: FOLLOWER
I20260810 06:49:30.077751 8387 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ba45b3bed043b5bc5943b63d0e3f7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46753 } }
I20260810 06:49:30.077802 8387 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:30.077823 8387 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:30.077857 8387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:30.078367 8387 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ba45b3bed043b5bc5943b63d0e3f7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46753 } }
I20260810 06:49:30.078454 8387 leader_election.cc:302] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6ba45b3bed043b5bc5943b63d0e3f7a; no voters:
I20260810 06:49:30.078586 8387 leader_election.cc:288] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:30.078644 8390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:30.078804 8390 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [term 1 LEADER]: Becoming Leader. State: Replica: c6ba45b3bed043b5bc5943b63d0e3f7a, State: Running, Role: LEADER
I20260810 06:49:30.078850 8387 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:30.078938 8390 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ba45b3bed043b5bc5943b63d0e3f7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46753 } }
I20260810 06:49:30.079514 8391 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6ba45b3bed043b5bc5943b63d0e3f7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ba45b3bed043b5bc5943b63d0e3f7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46753 } } }
I20260810 06:49:30.079648 8392 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6ba45b3bed043b5bc5943b63d0e3f7a. Latest consensus state: current_term: 1 leader_uuid: "c6ba45b3bed043b5bc5943b63d0e3f7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ba45b3bed043b5bc5943b63d0e3f7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46753 } } }
I20260810 06:49:30.079793 8392 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:30.079725 8391 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:30.080056 8401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:30.080597 8401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:30.080946 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 8307
I20260810 06:49:30.081079 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:30.081313 16291 external_mini_cluster.cc:1428] Setting key 02788e2b8ddde99259e88c0091836bc5
I20260810 06:49:30.082378 8401 catalog_manager.cc:1347] Generated new cluster ID: 947e52b5aa9845c59931e6124e0ccb8f
I20260810 06:49:30.082433 8401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:49:30 dist-test-slave-vxj2 krb5kdc[8292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:49:30.093902 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.081835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:36013 (local address 127.0.0.1:46753)
0810 06:49:30.082216 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.082226 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.082259 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:49:30.082393 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.082400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.082460 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.082600 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.083364 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.084906 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.085967 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.086669 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.086791 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.089866 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:30.089873 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:30.089873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:30.089888 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:30.091934 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:30.092437 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:30.092441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:30.092441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:30.092562 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:30.092817 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:30.092818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:30.092818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:30.093182 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:30.093342 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.093521 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.093633 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":95,"threads_started":1}
I20260810 06:49:30.094897 8401 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:30.095031 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:30.095584 8401 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:30.104333 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.096697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:36383 (local address 127.0.0.1:46753)
0810 06:49:30.096734 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.096736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.096746 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:49:30.096793 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.096794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.096820 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.096879 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.097535 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.098315 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.099324 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.099678 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.099752 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.101224 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:30.101226 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:30.101226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:30.101234 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:30.102972 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:30.103404 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:30.103405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:30.103405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:30.103470 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:30.103693 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:30.103694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:30.103695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:30.103831 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:30.103951 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.104052 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.104117 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:49:30.106936 8401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6ba45b3bed043b5bc5943b63d0e3f7a: Generated new TSK 0
I20260810 06:49:30.107163 8401 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:30.111558 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 8307
W20260810 06:49:30.122850 8289 connection.cc:535] client connection to 127.0.0.1:46753 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46753): Connection reset by peer (error 104)
2026-08-10T06:49:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:30 dist-test-slave-vxj2 krb5kdc[8423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:30 dist-test-slave-vxj2 krb5kdc[8423](info): set up 2 sockets
Aug 10 06:49:30 dist-test-slave-vxj2 krb5kdc[8423](info): commencing operation
krb5kdc: starting...
W20260810 06:49:31.295625 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:31 dist-test-slave-vxj2 krb5kdc[8423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:31.433115 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34793
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:31.522217 8440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:31.522447 8440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:31.522475 8440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:31.523664 8440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:31.523723 8440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:31.523741 8440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:31.523797 8440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:31.523824 8440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:31.525534 8440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34793
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:31.525812 8440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:31.526064 8440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:31.531405 8451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:31.531489 8440 server_base.cc:1034] running on GCE node
W20260810 06:49:31.531424 8447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:31.531433 8446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:31.531956 8440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:31.532296 8440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:31.533464 8440 hybrid_clock.cc:648] HybridClock initialized: now 1786344571533443 us; error 39 us; skew 500 ppm
Aug 10 06:49:31 dist-test-slave-vxj2 krb5kdc[8423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344571, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:31.535983 8440 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:31.536446 8440 webserver.cc:460] Webserver started at http://127.0.0.1:43041/ using document root <none> and password file <none>
I20260810 06:49:31.536696 8440 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:31.536764 8440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:31.536909 8440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:31.538072 8440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "d82a1395a948461f8220d1665e548489"
format_stamp: "Formatted at 2026-08-10 06:49:31 on dist-test-slave-vxj2"
server_key: "9d1576a5db252097baef5d4609866e9f"
server_key_iv: "eebcb0858df82196c2f9a7d5f6794bdd"
server_key_version: "encryptionkey@0"
I20260810 06:49:31.538455 8440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "d82a1395a948461f8220d1665e548489"
format_stamp: "Formatted at 2026-08-10 06:49:31 on dist-test-slave-vxj2"
server_key: "9d1576a5db252097baef5d4609866e9f"
server_key_iv: "eebcb0858df82196c2f9a7d5f6794bdd"
server_key_version: "encryptionkey@0"
I20260810 06:49:31.539821 8440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:49:31.540570 8460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:31.540740 8440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:49:31.540848 8440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "d82a1395a948461f8220d1665e548489"
format_stamp: "Formatted at 2026-08-10 06:49:31 on dist-test-slave-vxj2"
server_key: "9d1576a5db252097baef5d4609866e9f"
server_key_iv: "eebcb0858df82196c2f9a7d5f6794bdd"
server_key_version: "encryptionkey@0"
I20260810 06:49:31.540925 8440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:31.572471 8440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:31.574928 8440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:31.575103 8440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:31.579483 8440 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34793
I20260810 06:49:31.579543 8527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34793 every 8 connection(s)
I20260810 06:49:31.579970 8440 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:31.580974 8528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:31.583425 8528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489: Bootstrap starting.
I20260810 06:49:31.583993 8528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:31.584224 8528 log.cc:824] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:31.584964 8528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489: No bootstrap required, opened a new log
I20260810 06:49:31.586570 8528 raft_consensus.cc:348] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82a1395a948461f8220d1665e548489" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34793 } }
I20260810 06:49:31.586715 8528 raft_consensus.cc:374] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:31.586738 8528 raft_consensus.cc:729] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d82a1395a948461f8220d1665e548489, State: Initialized, Role: FOLLOWER
I20260810 06:49:31.586897 8528 consensus_queue.cc:260] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82a1395a948461f8220d1665e548489" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34793 } }
I20260810 06:49:31.586982 8528 raft_consensus.cc:388] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:31.587006 8528 raft_consensus.cc:482] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:31.587035 8528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:31.587548 8528 raft_consensus.cc:504] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82a1395a948461f8220d1665e548489" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34793 } }
I20260810 06:49:31.587638 8528 leader_election.cc:302] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d82a1395a948461f8220d1665e548489; no voters:
I20260810 06:49:31.587786 8528 leader_election.cc:288] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:31.587868 8532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:31.588042 8528 sys_catalog.cc:564] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:31.588047 8532 raft_consensus.cc:686] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [term 1 LEADER]: Becoming Leader. State: Replica: d82a1395a948461f8220d1665e548489, State: Running, Role: LEADER
I20260810 06:49:31.588296 8532 consensus_queue.cc:237] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82a1395a948461f8220d1665e548489" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34793 } }
I20260810 06:49:31.588912 8534 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d82a1395a948461f8220d1665e548489. Latest consensus state: current_term: 1 leader_uuid: "d82a1395a948461f8220d1665e548489" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82a1395a948461f8220d1665e548489" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34793 } } }
I20260810 06:49:31.589008 8534 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:31.589120 8533 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d82a1395a948461f8220d1665e548489" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82a1395a948461f8220d1665e548489" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34793 } } }
I20260810 06:49:31.589255 8533 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:31.589259 8543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:31.589481 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 8440
I20260810 06:49:31.589617 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:31.589781 8543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:31.589871 16291 external_mini_cluster.cc:1428] Setting key b73f5c8ff10f0abd90c5776c23ac44b5
I20260810 06:49:31.591899 8543 catalog_manager.cc:1347] Generated new cluster ID: 0e0fbe3bfc8847e49e8a4dfc47b7c87d
I20260810 06:49:31.591953 8543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:31.596302 8543 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:31.597076 8543 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:31.601611 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:31.590556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:41125 (local address 127.0.0.1:34793)
0810 06:49:31.590966 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:31.590975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:31.591010 (+ 35us) server_negotiation.cc:407] Connection header received
0810 06:49:31.591137 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:31.591144 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:31.591218 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:31.591362 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:31.591589 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:31.592979 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:31.594128 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:31.594770 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:31.595204 (+ 434us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:31.595724 (+ 520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:31.595728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:31.595728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:31.595959 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:31.596006 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:31.596023 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:31.596109 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:31.601297 (+ 5188us) server_negotiation.cc:299] Negotiation successful
0810 06:49:31.601334 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":75,"threads_started":1}
I20260810 06:49:31.602708 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:31.604096 8543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d82a1395a948461f8220d1665e548489: Generated new TSK 0
I20260810 06:49:31.604326 8543 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:31.607756 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:31.604113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:39013 (local address 127.0.0.1:34793)
0810 06:49:31.604155 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:31.604157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:31.604396 (+ 239us) server_negotiation.cc:407] Connection header received
0810 06:49:31.604492 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:31.604493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:31.604506 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:31.604566 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:31.604682 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:31.605553 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:31.606312 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:31.606643 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:31.606715 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:31.607482 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:31.607483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:31.607483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:31.607489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:31.607506 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:31.607511 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:31.607575 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:31.607647 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 06:49:31.607662 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:49:31.609786 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 8440
W20260810 06:49:31.617014 8419 connection.cc:535] client connection to 127.0.0.1:34793 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34793): Connection reset by peer (error 104)
2026-08-10T06:49:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:31 dist-test-slave-vxj2 krb5kdc[8562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:31 dist-test-slave-vxj2 krb5kdc[8562](info): set up 2 sockets
Aug 10 06:49:31 dist-test-slave-vxj2 krb5kdc[8562](info): commencing operation
krb5kdc: starting...
W20260810 06:49:32.751973 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:32 dist-test-slave-vxj2 krb5kdc[8562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:32.876654 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40605
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:32.959251 8576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:32.959493 8576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:32.959532 8576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:32.960660 8576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:32.960727 8576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:32.960750 8576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:32.960772 8576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:32.960799 8576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:32.962277 8576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40605
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:32.962555 8576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:32.962785 8576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:32.968422 8583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:32.968416 8585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:32.968416 8582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:32.968604 8576 server_base.cc:1034] running on GCE node
I20260810 06:49:32.968966 8576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:32.969305 8576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:32.970476 8576 hybrid_clock.cc:648] HybridClock initialized: now 1786344572970460 us; error 30 us; skew 500 ppm
Aug 10 06:49:32 dist-test-slave-vxj2 krb5kdc[8562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344572, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:32.973227 8576 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:32.973716 8576 webserver.cc:460] Webserver started at http://127.0.0.1:40315/ using document root <none> and password file <none>
I20260810 06:49:32.973943 8576 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:32.973987 8576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:32.974112 8576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:32.975705 8576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "620a429de12e4108b7897adb9551c4b9"
format_stamp: "Formatted at 2026-08-10 06:49:32 on dist-test-slave-vxj2"
server_key: "a8b60d38d04e2d6367f6277a0a15d756"
server_key_iv: "02f7a160c46d55ec6c7c1ec0974079dd"
server_key_version: "encryptionkey@0"
I20260810 06:49:32.976114 8576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "620a429de12e4108b7897adb9551c4b9"
format_stamp: "Formatted at 2026-08-10 06:49:32 on dist-test-slave-vxj2"
server_key: "a8b60d38d04e2d6367f6277a0a15d756"
server_key_iv: "02f7a160c46d55ec6c7c1ec0974079dd"
server_key_version: "encryptionkey@0"
I20260810 06:49:32.977509 8576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:32.978305 8593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:32.978503 8576 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:32.978598 8576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "620a429de12e4108b7897adb9551c4b9"
format_stamp: "Formatted at 2026-08-10 06:49:32 on dist-test-slave-vxj2"
server_key: "a8b60d38d04e2d6367f6277a0a15d756"
server_key_iv: "02f7a160c46d55ec6c7c1ec0974079dd"
server_key_version: "encryptionkey@0"
I20260810 06:49:32.978660 8576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:32.982195 8576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:32.984558 8576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:32.984683 8576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:32.988759 8576 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40605
I20260810 06:49:32.988821 8659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40605 every 8 connection(s)
I20260810 06:49:32.989214 8576 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:32.990135 8660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:32.991853 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 8576
I20260810 06:49:32.991993 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:32.992189 16291 external_mini_cluster.cc:1428] Setting key 829c2712fa6407494ddc0d50203ffd7c
I20260810 06:49:32.992702 8660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9: Bootstrap starting.
I20260810 06:49:32.993396 8660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:32.993743 8660 log.cc:824] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:32.994889 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:32.992756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:53151 (local address 127.0.0.1:40605)
0810 06:49:32.992920 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:32.992928 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:32.992953 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:49:32.993442 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:32.993450 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:32.993496 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:32.993641 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:32.994352 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:32.994356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:32.994357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:32.994361 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:32.994407 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:32.994421 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:32.994513 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:32.994572 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 06:49:32.994614 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260810 06:49:32.994897 8660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9: No bootstrap required, opened a new log
I20260810 06:49:32.996023 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:32.997123 8660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a429de12e4108b7897adb9551c4b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40605 } }
I20260810 06:49:32.997390 8660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:32.997437 8660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 620a429de12e4108b7897adb9551c4b9, State: Initialized, Role: FOLLOWER
I20260810 06:49:32.997563 8660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a429de12e4108b7897adb9551c4b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40605 } }
I20260810 06:49:32.997658 8660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:32.997750 8660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:32.997831 8660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:32.998697 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:32.997653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:45267 (local address 127.0.0.1:40605)
0810 06:49:32.997692 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:32.997694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:32.997704 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:49:32.997726 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:32.997726 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:32.997731 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:32.997814 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:32.998429 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:32.998431 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:32.998431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:32.998436 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:32.998452 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:32.998456 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:32.998522 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:32.998601 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 06:49:32.998616 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:49:32.998688 8660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a429de12e4108b7897adb9551c4b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40605 } }
I20260810 06:49:32.998814 8660 leader_election.cc:302] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 620a429de12e4108b7897adb9551c4b9; no voters:
I20260810 06:49:32.998991 8660 leader_election.cc:288] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:32.999046 8671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:32.999248 8660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:32.999286 8671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [term 1 LEADER]: Becoming Leader. State: Replica: 620a429de12e4108b7897adb9551c4b9, State: Running, Role: LEADER
I20260810 06:49:32.999377 8671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a429de12e4108b7897adb9551c4b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40605 } }
I20260810 06:49:32.999760 8673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "620a429de12e4108b7897adb9551c4b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a429de12e4108b7897adb9551c4b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40605 } } }
I20260810 06:49:32.999773 8674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 620a429de12e4108b7897adb9551c4b9. Latest consensus state: current_term: 1 leader_uuid: "620a429de12e4108b7897adb9551c4b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a429de12e4108b7897adb9551c4b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40605 } } }
I20260810 06:49:32.999899 8673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:32.999972 8674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:33.000501 8681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:33.001000 8681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:33.002548 8681 catalog_manager.cc:1347] Generated new cluster ID: 50e254305b6449b18f8f16bddad0ef8e
I20260810 06:49:33.002689 8681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:33.054095 8681 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:33.054319 8681 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:33.063220 8681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 620a429de12e4108b7897adb9551c4b9: Generated new TSK 0
I20260810 06:49:33.063447 8681 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:33.067756 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 8576
2026-08-10T06:49:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1460 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:33 dist-test-slave-vxj2 krb5kdc[8697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:33 dist-test-slave-vxj2 krb5kdc[8697](info): set up 2 sockets
Aug 10 06:49:33 dist-test-slave-vxj2 krb5kdc[8697](info): commencing operation
krb5kdc: starting...
W20260810 06:49:34.188002 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:34 dist-test-slave-vxj2 krb5kdc[8697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:49:34.317807 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:42957
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:42957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:34.404915 8712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:34.405257 8712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:34.405313 8712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:34.406656 8712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:34.406718 8712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:34.406736 8712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:34.406752 8712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:34.406767 8712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:34.408484 8712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:42957
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:42957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:34.408792 8712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:34.409047 8712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:34.414623 8722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:34.414598 8718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:34.414588 8717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:34.415155 8712 server_base.cc:1034] running on GCE node
I20260810 06:49:34.415472 8712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:34.415813 8712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:34.416994 8712 hybrid_clock.cc:648] HybridClock initialized: now 1786344574416980 us; error 34 us; skew 500 ppm
Aug 10 06:49:34 dist-test-slave-vxj2 krb5kdc[8697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344574, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:34.423637 8712 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:49:34.424243 8712 webserver.cc:460] Webserver started at http://127.15.232.254:43627/ using document root <none> and password file <none>
I20260810 06:49:34.424494 8712 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:34.424541 8712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:34.424715 8712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:34.425961 8712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "9ce9076a7377447e8ca89de934f10b68"
format_stamp: "Formatted at 2026-08-10 06:49:34 on dist-test-slave-vxj2"
server_key: "23defe774cfb4ceebd7cc34697f0ff04"
server_key_iv: "40125f3bceca633a0db299eb1479304f"
server_key_version: "encryptionkey@0"
I20260810 06:49:34.426345 8712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "9ce9076a7377447e8ca89de934f10b68"
format_stamp: "Formatted at 2026-08-10 06:49:34 on dist-test-slave-vxj2"
server_key: "23defe774cfb4ceebd7cc34697f0ff04"
server_key_iv: "40125f3bceca633a0db299eb1479304f"
server_key_version: "encryptionkey@0"
I20260810 06:49:34.427870 8712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:49:34.428673 8731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:34.428902 8712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:34.429014 8712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "9ce9076a7377447e8ca89de934f10b68"
format_stamp: "Formatted at 2026-08-10 06:49:34 on dist-test-slave-vxj2"
server_key: "23defe774cfb4ceebd7cc34697f0ff04"
server_key_iv: "40125f3bceca633a0db299eb1479304f"
server_key_version: "encryptionkey@0"
I20260810 06:49:34.429090 8712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:34.450125 8712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:34.452710 8712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:34.452900 8712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:34.457108 8712 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:42957
I20260810 06:49:34.457288 8792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:42957 every 8 connection(s)
I20260810 06:49:34.457585 8712 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:34.458546 8793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:34.461143 8793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68: Bootstrap starting.
I20260810 06:49:34.461812 8793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:34.462126 8793 log.cc:824] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:34.462939 8793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68: No bootstrap required, opened a new log
I20260810 06:49:34.464004 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 8712
I20260810 06:49:34.464151 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:34.464396 16291 external_mini_cluster.cc:1428] Setting key 09f4d45d66d166c49756e96cbddad52e
I20260810 06:49:34.465443 8793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ce9076a7377447e8ca89de934f10b68" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42957 } }
I20260810 06:49:34.465646 8793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:34.465708 8793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ce9076a7377447e8ca89de934f10b68, State: Initialized, Role: FOLLOWER
I20260810 06:49:34.465885 8793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ce9076a7377447e8ca89de934f10b68" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42957 } }
I20260810 06:49:34.465988 8793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:34.466028 8793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:34.466095 8793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:34.467218 8793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ce9076a7377447e8ca89de934f10b68" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42957 } }
I20260810 06:49:34.467324 8793 leader_election.cc:302] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ce9076a7377447e8ca89de934f10b68; no voters:
I20260810 06:49:34.467561 8793 leader_election.cc:288] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:34.467581 8799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:34.467715 8799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [term 1 LEADER]: Becoming Leader. State: Replica: 9ce9076a7377447e8ca89de934f10b68, State: Running, Role: LEADER
I20260810 06:49:34.467878 8793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:34.467902 8799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ce9076a7377447e8ca89de934f10b68" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42957 } }
I20260810 06:49:34.468374 8800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ce9076a7377447e8ca89de934f10b68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ce9076a7377447e8ca89de934f10b68" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42957 } } }
I20260810 06:49:34.468469 8800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:34.468412 8801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ce9076a7377447e8ca89de934f10b68. Latest consensus state: current_term: 1 leader_uuid: "9ce9076a7377447e8ca89de934f10b68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ce9076a7377447e8ca89de934f10b68" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 42957 } } }
I20260810 06:49:34.468523 8801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:34.468699 8807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:34.469404 8807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:34.471341 8807 catalog_manager.cc:1347] Generated new cluster ID: 5b1c93b209cf454fbf88dd03e6686362
I20260810 06:49:34.471477 8807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:49:34 dist-test-slave-vxj2 krb5kdc[8697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:34.475991 8797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:34.465031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41769 (local address 127.15.232.254:42957)
0810 06:49:34.465245 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:34.465255 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:34.465294 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:49:34.465398 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:34.465405 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:34.465457 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:34.465573 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:34.466499 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:34.467578 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:34.468637 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:34.469340 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:34.469453 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:34.472424 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:34.472428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:34.472428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:34.472442 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:34.474253 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:34.474703 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:34.474706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:34.474707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:34.474833 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:34.475079 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:34.475080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:34.475081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:34.475352 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:34.475561 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:34.475652 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 06:49:34.475765 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":123,"threads_started":1}
I20260810 06:49:34.477067 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:34.488025 8797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:34.478552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54139 (local address 127.15.232.254:42957)
0810 06:49:34.478617 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:34.478620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:34.481421 (+ 2801us) server_negotiation.cc:407] Connection header received
0810 06:49:34.481507 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:34.481508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:34.481520 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:34.481586 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:34.482206 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:34.482746 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:34.483500 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:34.483743 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:34.483819 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:34.485197 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:34.485199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:34.485200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:34.485209 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:34.486753 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:34.487164 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:34.487165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:34.487166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:34.487227 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:34.487458 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:34.487459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:34.487460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:34.487582 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:34.487695 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:34.487828 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 06:49:34.487903 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:49:34.490715 8807 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:34.491291 8807 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:34.496948 8807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ce9076a7377447e8ca89de934f10b68: Generated new TSK 0
I20260810 06:49:34.497198 8807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:34.499557 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 8712
W20260810 06:49:34.507578 8695 connection.cc:535] client connection to 127.15.232.254:42957 recv error: Network error: failed to read from TLS socket (remote: 127.15.232.254:42957): Connection reset by peer (error 104)
2026-08-10T06:49:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:34 dist-test-slave-vxj2 krb5kdc[8835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:34 dist-test-slave-vxj2 krb5kdc[8835](info): set up 2 sockets
Aug 10 06:49:34 dist-test-slave-vxj2 krb5kdc[8835](info): commencing operation
krb5kdc: starting...
W20260810 06:49:35.659183 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:35 dist-test-slave-vxj2 krb5kdc[8835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:49:35.788653 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:43355
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:36521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:43355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:35.873324 8852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:35.873608 8852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:35.873646 8852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:35.874872 8852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:35.874954 8852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:35.874979 8852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:35.875002 8852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:35.875046 8852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:35.876629 8852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:36521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:43355
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:43355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:35.876937 8852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:35.877220 8852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:35.882416 8858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:35.882421 8857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:35.882416 8861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:35.882692 8852 server_base.cc:1034] running on GCE node
I20260810 06:49:35.883054 8852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:35.883351 8852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:35.884534 8852 hybrid_clock.cc:648] HybridClock initialized: now 1786344575884513 us; error 35 us; skew 500 ppm
Aug 10 06:49:35 dist-test-slave-vxj2 krb5kdc[8835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:35.887256 8852 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:49:35.887753 8852 webserver.cc:460] Webserver started at http://127.15.232.254:44503/ using document root <none> and password file <none>
I20260810 06:49:35.888013 8852 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:35.888085 8852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:35.888234 8852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:35.889565 8852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "5884de9054834ae9acbfb5db99b3ad76"
format_stamp: "Formatted at 2026-08-10 06:49:35 on dist-test-slave-vxj2"
server_key: "06ed3fbc49166c00cfe7e3996d981889"
server_key_iv: "8e513c64129f89c7815bc220fbaa99d8"
server_key_version: "encryptionkey@0"
I20260810 06:49:35.889947 8852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "5884de9054834ae9acbfb5db99b3ad76"
format_stamp: "Formatted at 2026-08-10 06:49:35 on dist-test-slave-vxj2"
server_key: "06ed3fbc49166c00cfe7e3996d981889"
server_key_iv: "8e513c64129f89c7815bc220fbaa99d8"
server_key_version: "encryptionkey@0"
I20260810 06:49:35.891435 8852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:35.892351 8869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:35.892578 8852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:35.892689 8852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "5884de9054834ae9acbfb5db99b3ad76"
format_stamp: "Formatted at 2026-08-10 06:49:35 on dist-test-slave-vxj2"
server_key: "06ed3fbc49166c00cfe7e3996d981889"
server_key_iv: "8e513c64129f89c7815bc220fbaa99d8"
server_key_version: "encryptionkey@0"
I20260810 06:49:35.892767 8852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:35.903746 8852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:35.906275 8852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:35.906472 8852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:35.910889 8852 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:43355
I20260810 06:49:35.910931 8930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:43355 every 8 connection(s)
I20260810 06:49:35.911314 8852 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:35.912258 8931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:35.914568 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 8852
I20260810 06:49:35.914707 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:35.914909 16291 external_mini_cluster.cc:1428] Setting key 2cc71596633c462ae5cdc9b347b232a3
I20260810 06:49:35.915210 8931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76: Bootstrap starting.
I20260810 06:49:35.915913 8931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:35.916204 8931 log.cc:824] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:35.916900 8931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76: No bootstrap required, opened a new log
I20260810 06:49:35.918758 8931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5884de9054834ae9acbfb5db99b3ad76" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43355 } }
I20260810 06:49:35.918926 8931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:35.918965 8931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5884de9054834ae9acbfb5db99b3ad76, State: Initialized, Role: FOLLOWER
I20260810 06:49:35.919121 8931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5884de9054834ae9acbfb5db99b3ad76" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43355 } }
I20260810 06:49:35.919216 8931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:35.919265 8931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:35.919358 8931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:35.919930 8931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5884de9054834ae9acbfb5db99b3ad76" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43355 } }
I20260810 06:49:35.920049 8931 leader_election.cc:302] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5884de9054834ae9acbfb5db99b3ad76; no voters:
I20260810 06:49:35.920254 8931 leader_election.cc:288] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:35.920315 8937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:35.920506 8937 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [term 1 LEADER]: Becoming Leader. State: Replica: 5884de9054834ae9acbfb5db99b3ad76, State: Running, Role: LEADER
I20260810 06:49:35.920611 8931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:35.920673 8937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5884de9054834ae9acbfb5db99b3ad76" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43355 } }
I20260810 06:49:35.920853 8935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:35.915459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38983 (local address 127.15.232.254:43355)
0810 06:49:35.915648 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:35.915659 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:35.915701 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:49:35.916226 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:35.916234 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:35.916301 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:35.916468 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:35.916661 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:35.917683 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:35.918486 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:35.918914 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:35.919035 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:35.919835 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:35.919839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:35.919840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:35.920020 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:35.920076 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:35.920091 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:35.920180 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:35.920495 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 06:49:35.920531 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260810 06:49:35.921108 8937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5884de9054834ae9acbfb5db99b3ad76. Latest consensus state: current_term: 1 leader_uuid: "5884de9054834ae9acbfb5db99b3ad76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5884de9054834ae9acbfb5db99b3ad76" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43355 } } }
I20260810 06:49:35.921351 8937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:35.921280 8938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5884de9054834ae9acbfb5db99b3ad76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5884de9054834ae9acbfb5db99b3ad76" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43355 } } }
I20260810 06:49:35.921567 8938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:35.921598 8945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:35.922019 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:35.922125 8945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:35.924525 8945 catalog_manager.cc:1347] Generated new cluster ID: 2bbd5e5779804cca9a02da2f807037ea
I20260810 06:49:35.924580 8945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:35.927464 8935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:35.923623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59349 (local address 127.15.232.254:43355)
0810 06:49:35.923662 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:35.923664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:35.923904 (+ 240us) server_negotiation.cc:407] Connection header received
0810 06:49:35.923958 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:35.923959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:35.923969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:35.924038 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:35.924238 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:35.925002 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:35.926045 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:35.926370 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:35.926449 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:35.927201 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:35.927202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:35.927203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:35.927208 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:35.927224 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:35.927229 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:35.927299 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:35.927374 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 06:49:35.927386 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:49:35.939461 8945 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:35.940373 8945 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:35.947827 8945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5884de9054834ae9acbfb5db99b3ad76: Generated new TSK 0
I20260810 06:49:35.948065 8945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:35.952502 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 8852
W20260810 06:49:35.960595 8826 connection.cc:535] client connection to 127.15.232.254:43355 recv error: Network error: failed to read from TLS socket (remote: 127.15.232.254:43355): Connection reset by peer (error 104)
2026-08-10T06:49:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1453 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:36 dist-test-slave-vxj2 krb5kdc[8967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:36 dist-test-slave-vxj2 krb5kdc[8967](info): set up 2 sockets
Aug 10 06:49:36 dist-test-slave-vxj2 krb5kdc[8967](info): commencing operation
krb5kdc: starting...
W20260810 06:49:37.079149 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:37 dist-test-slave-vxj2 krb5kdc[8967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:49:37.229065 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:36691
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:36691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:37.314925 8984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:37.315209 8984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:37.315256 8984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:37.316512 8984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:37.316588 8984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:37.316617 8984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:37.316648 8984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:37.316684 8984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:37.318264 8984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:36691
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:36691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:37.318570 8984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:37.318827 8984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:37.324182 8994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:37.324293 8984 server_base.cc:1034] running on GCE node
W20260810 06:49:37.324201 8992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:37.324389 8991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:37.324668 8984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:37.325078 8984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:37.326239 8984 hybrid_clock.cc:648] HybridClock initialized: now 1786344577326213 us; error 36 us; skew 500 ppm
Aug 10 06:49:37 dist-test-slave-vxj2 krb5kdc[8967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344577, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:37.329053 8984 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:49:37.329617 8984 webserver.cc:460] Webserver started at http://127.15.232.254:36369/ using document root <none> and password file <none>
I20260810 06:49:37.329998 8984 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:37.330060 8984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:37.330233 8984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:37.331341 8984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "1ff9cdc53a8540e39c12be32279b4220"
format_stamp: "Formatted at 2026-08-10 06:49:37 on dist-test-slave-vxj2"
server_key: "b8f0a01f5b4b40f0f8f2a5332d1f9f21"
server_key_iv: "628647940360a0ab388cb1dc29098373"
server_key_version: "encryptionkey@0"
I20260810 06:49:37.331728 8984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "1ff9cdc53a8540e39c12be32279b4220"
format_stamp: "Formatted at 2026-08-10 06:49:37 on dist-test-slave-vxj2"
server_key: "b8f0a01f5b4b40f0f8f2a5332d1f9f21"
server_key_iv: "628647940360a0ab388cb1dc29098373"
server_key_version: "encryptionkey@0"
I20260810 06:49:37.333216 8984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:49:37.334038 9003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:37.334313 8984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:49:37.334395 8984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "1ff9cdc53a8540e39c12be32279b4220"
format_stamp: "Formatted at 2026-08-10 06:49:37 on dist-test-slave-vxj2"
server_key: "b8f0a01f5b4b40f0f8f2a5332d1f9f21"
server_key_iv: "628647940360a0ab388cb1dc29098373"
server_key_version: "encryptionkey@0"
I20260810 06:49:37.334493 8984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:37.338014 8984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:37.340405 8984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:37.340570 8984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:37.345211 8984 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:36691
I20260810 06:49:37.345263 9062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:36691 every 8 connection(s)
I20260810 06:49:37.345623 8984 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:37.346624 9063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:37.348974 9063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220: Bootstrap starting.
I20260810 06:49:37.349618 9063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:37.349920 9063 log.cc:824] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:37.350656 9063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220: No bootstrap required, opened a new log
I20260810 06:49:37.352262 9063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff9cdc53a8540e39c12be32279b4220" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36691 } }
I20260810 06:49:37.352411 9063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:37.352434 9063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ff9cdc53a8540e39c12be32279b4220, State: Initialized, Role: FOLLOWER
I20260810 06:49:37.352588 9063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff9cdc53a8540e39c12be32279b4220" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36691 } }
I20260810 06:49:37.352651 9063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:37.352691 9063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:37.352741 9063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:37.353657 9063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff9cdc53a8540e39c12be32279b4220" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36691 } }
I20260810 06:49:37.353747 9063 leader_election.cc:302] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ff9cdc53a8540e39c12be32279b4220; no voters:
I20260810 06:49:37.353919 9063 leader_election.cc:288] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:37.353989 9066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:37.354220 9063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:37.354204 9066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [term 1 LEADER]: Becoming Leader. State: Replica: 1ff9cdc53a8540e39c12be32279b4220, State: Running, Role: LEADER
I20260810 06:49:37.354420 9066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff9cdc53a8540e39c12be32279b4220" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36691 } }
I20260810 06:49:37.354727 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 8984
I20260810 06:49:37.354872 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:37.354844 9067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ff9cdc53a8540e39c12be32279b4220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff9cdc53a8540e39c12be32279b4220" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36691 } } }
I20260810 06:49:37.354918 9067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:37.355115 16291 external_mini_cluster.cc:1428] Setting key 92da8a3571616adad2d88f190735b50b
I20260810 06:49:37.355096 9069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ff9cdc53a8540e39c12be32279b4220. Latest consensus state: current_term: 1 leader_uuid: "1ff9cdc53a8540e39c12be32279b4220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff9cdc53a8540e39c12be32279b4220" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 36691 } } }
I20260810 06:49:37.355150 9069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:37.355469 9076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:37.356086 9076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:37.358109 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:37.355652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33415 (local address 127.15.232.254:36691)
0810 06:49:37.355825 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:37.355836 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:37.355889 (+ 53us) server_negotiation.cc:407] Connection header received
0810 06:49:37.356101 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:37.356111 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:37.356150 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:37.356423 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:37.357632 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:37.357636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:37.357636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:37.357641 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:37.357706 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:37.357719 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:37.357801 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:37.357883 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:49:37.357932 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260810 06:49:37.359230 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:37.361692 9076 catalog_manager.cc:1347] Generated new cluster ID: bb2985586da844c1af573dd9b8a2c9e6
I20260810 06:49:37.361761 9076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:37.370003 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:37.360725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55413 (local address 127.15.232.254:36691)
0810 06:49:37.360788 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:37.360790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:37.361858 (+ 1068us) server_negotiation.cc:407] Connection header received
0810 06:49:37.361883 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:37.361884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:37.361890 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:37.361935 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:37.369698 (+ 7763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:37.369700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:37.369701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:37.369708 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:37.369741 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:37.369747 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:37.369799 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:37.369886 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 06:49:37.369905 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:49:37.373099 9076 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:37.373343 9076 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:37.378548 9076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ff9cdc53a8540e39c12be32279b4220: Generated new TSK 0
I20260810 06:49:37.378784 9076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:37.381547 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 8984
2026-08-10T06:49:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:37 dist-test-slave-vxj2 krb5kdc[9099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:37 dist-test-slave-vxj2 krb5kdc[9099](info): set up 2 sockets
Aug 10 06:49:37 dist-test-slave-vxj2 krb5kdc[9099](info): commencing operation
krb5kdc: starting...
W20260810 06:49:38.515273 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:38 dist-test-slave-vxj2 krb5kdc[9099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:49:38.646538 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:39861
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:39861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:38.730288 9115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:38.730517 9115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:38.730541 9115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:38.731649 9115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:38.731686 9115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:38.731701 9115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:38.731714 9115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:38.731725 9115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:38.733174 9115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:39861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:39861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:38.733381 9115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:38.733560 9115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:38.738574 9120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:38.738602 9123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:38.738648 9121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:38.738672 9115 server_base.cc:1034] running on GCE node
I20260810 06:49:38.739113 9115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:38.739421 9115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:38.740585 9115 hybrid_clock.cc:648] HybridClock initialized: now 1786344578740581 us; error 41 us; skew 500 ppm
Aug 10 06:49:38 dist-test-slave-vxj2 krb5kdc[9099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344578, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:38.743232 9115 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:49:38.743744 9115 webserver.cc:460] Webserver started at http://127.15.232.254:40693/ using document root <none> and password file <none>
I20260810 06:49:38.743983 9115 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:38.744052 9115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:38.744211 9115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:38.745422 9115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "d7b6194e1d914daeb528eba022c57b46"
format_stamp: "Formatted at 2026-08-10 06:49:38 on dist-test-slave-vxj2"
server_key: "f0ae8d8007469c9e64b950e049213cad"
server_key_iv: "c729c822426926609626b2bb68c15148"
server_key_version: "encryptionkey@0"
I20260810 06:49:38.745805 9115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "d7b6194e1d914daeb528eba022c57b46"
format_stamp: "Formatted at 2026-08-10 06:49:38 on dist-test-slave-vxj2"
server_key: "f0ae8d8007469c9e64b950e049213cad"
server_key_iv: "c729c822426926609626b2bb68c15148"
server_key_version: "encryptionkey@0"
I20260810 06:49:38.747339 9115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:49:38.748168 9131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:38.748386 9115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:38.748493 9115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "d7b6194e1d914daeb528eba022c57b46"
format_stamp: "Formatted at 2026-08-10 06:49:38 on dist-test-slave-vxj2"
server_key: "f0ae8d8007469c9e64b950e049213cad"
server_key_iv: "c729c822426926609626b2bb68c15148"
server_key_version: "encryptionkey@0"
I20260810 06:49:38.748584 9115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:38.765990 9115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:38.768460 9115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:38.768656 9115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:38.773180 9115 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:39861
I20260810 06:49:38.773608 9115 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:38.773651 9199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:39861 every 8 connection(s)
I20260810 06:49:38.774426 9200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:38.776950 9200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46: Bootstrap starting.
I20260810 06:49:38.777599 9200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:38.777908 9200 log.cc:824] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:38.778631 9200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46: No bootstrap required, opened a new log
I20260810 06:49:38.780329 9200 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b6194e1d914daeb528eba022c57b46" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39861 } }
I20260810 06:49:38.780484 9200 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:38.780508 9200 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b6194e1d914daeb528eba022c57b46, State: Initialized, Role: FOLLOWER
I20260810 06:49:38.780691 9200 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b6194e1d914daeb528eba022c57b46" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39861 } }
I20260810 06:49:38.780766 9200 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:38.780788 9200 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:38.780843 9200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:38.781891 9200 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b6194e1d914daeb528eba022c57b46" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39861 } }
I20260810 06:49:38.782014 9200 leader_election.cc:302] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7b6194e1d914daeb528eba022c57b46; no voters:
I20260810 06:49:38.782215 9200 leader_election.cc:288] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:38.782280 9203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:38.782492 9203 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [term 1 LEADER]: Becoming Leader. State: Replica: d7b6194e1d914daeb528eba022c57b46, State: Running, Role: LEADER
I20260810 06:49:38.782567 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 9115
I20260810 06:49:38.782529 9200 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:38.782698 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:38.782683 9203 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b6194e1d914daeb528eba022c57b46" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39861 } }
I20260810 06:49:38.782917 16291 external_mini_cluster.cc:1428] Setting key da84a7aa2d6cb6b44e937aca630b1687
I20260810 06:49:38.783080 9203 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7b6194e1d914daeb528eba022c57b46. Latest consensus state: current_term: 1 leader_uuid: "d7b6194e1d914daeb528eba022c57b46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b6194e1d914daeb528eba022c57b46" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39861 } } }
I20260810 06:49:38.783107 9205 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7b6194e1d914daeb528eba022c57b46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b6194e1d914daeb528eba022c57b46" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 39861 } } }
I20260810 06:49:38.783195 9203 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:38.783218 9205 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:38.783455 9212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:38.784081 9212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:38.785820 9212 catalog_manager.cc:1347] Generated new cluster ID: 9a57fdcdf397425f81632cb45865425f
I20260810 06:49:38.785878 9212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:49:38 dist-test-slave-vxj2 krb5kdc[9099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:38.794533 9214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:38.783434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59653 (local address 127.15.232.254:39861)
0810 06:49:38.783604 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:38.783616 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:38.784738 (+ 1122us) server_negotiation.cc:407] Connection header received
0810 06:49:38.784915 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:38.784921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:38.784969 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:38.785119 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:38.786040 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:38.787089 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:38.787885 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:38.788327 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:38.788451 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:38.790996 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:38.791001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:38.791001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:38.791014 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:38.792792 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:38.793207 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:38.793211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:38.793212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:38.793330 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:38.793558 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:38.793559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:38.793559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:38.793840 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:38.794029 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:38.794112 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 06:49:38.794270 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
I20260810 06:49:38.795522 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:38.803464 9214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:38.797047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50745 (local address 127.15.232.254:39861)
0810 06:49:38.797089 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:38.797091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:38.797319 (+ 228us) server_negotiation.cc:407] Connection header received
0810 06:49:38.797384 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:38.797385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:38.797397 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:38.797453 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:38.798009 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:38.798515 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:38.799224 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:38.799452 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:38.799538 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:38.800765 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:38.800767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:38.800768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:38.800775 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:38.802254 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:38.802621 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:38.802623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:38.802623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:38.802684 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:38.802904 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:38.802905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:38.802905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:38.803029 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:38.803131 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:38.803271 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 06:49:38.803351 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:49:38.812386 9212 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:38.813081 9212 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:38.824441 9212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7b6194e1d914daeb528eba022c57b46: Generated new TSK 0
I20260810 06:49:38.824717 9212 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:38.827433 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 9115
2026-08-10T06:49:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:38 dist-test-slave-vxj2 krb5kdc[9236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:38 dist-test-slave-vxj2 krb5kdc[9236](info): set up 2 sockets
Aug 10 06:49:38 dist-test-slave-vxj2 krb5kdc[9236](info): commencing operation
krb5kdc: starting...
W20260810 06:49:40.024029 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:40 dist-test-slave-vxj2 krb5kdc[9236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:49:40.160773 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:43799
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:44137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:43799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:40.247463 9249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:40.247725 9249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:40.247753 9249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:40.249037 9249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:40.249089 9249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:40.249104 9249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:40.249116 9249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:40.249128 9249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:40.250916 9249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:44137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:43799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:43799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:40.251237 9249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:40.251498 9249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:40.256803 9255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:40.256883 9258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:40.256919 9256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:40.257191 9249 server_base.cc:1034] running on GCE node
I20260810 06:49:40.257473 9249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:40.257812 9249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:40.259003 9249 hybrid_clock.cc:648] HybridClock initialized: now 1786344580258989 us; error 35 us; skew 500 ppm
Aug 10 06:49:40 dist-test-slave-vxj2 krb5kdc[9236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344580, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:40.267510 9249 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:49:40.268080 9249 webserver.cc:460] Webserver started at http://127.15.232.254:41295/ using document root <none> and password file <none>
I20260810 06:49:40.268352 9249 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:40.268401 9249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:40.268568 9249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:40.269788 9249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "ec0aa7636ebe417b8f6fb389313bfff3"
format_stamp: "Formatted at 2026-08-10 06:49:40 on dist-test-slave-vxj2"
server_key: "bdea165907d79281431a52b219121650"
server_key_iv: "7c8d569201e3f5071036c75e153c45c2"
server_key_version: "encryptionkey@0"
I20260810 06:49:40.270179 9249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "ec0aa7636ebe417b8f6fb389313bfff3"
format_stamp: "Formatted at 2026-08-10 06:49:40 on dist-test-slave-vxj2"
server_key: "bdea165907d79281431a52b219121650"
server_key_iv: "7c8d569201e3f5071036c75e153c45c2"
server_key_version: "encryptionkey@0"
I20260810 06:49:40.271693 9249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:49:40.272575 9265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:40.272778 9249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:40.272889 9249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "ec0aa7636ebe417b8f6fb389313bfff3"
format_stamp: "Formatted at 2026-08-10 06:49:40 on dist-test-slave-vxj2"
server_key: "bdea165907d79281431a52b219121650"
server_key_iv: "7c8d569201e3f5071036c75e153c45c2"
server_key_version: "encryptionkey@0"
I20260810 06:49:40.272989 9249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:40.296147 9249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:40.298902 9249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:40.299151 9249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:40.304236 9249 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:43799
I20260810 06:49:40.304286 9326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:43799 every 8 connection(s)
I20260810 06:49:40.304684 9249 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:40.305673 9328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:40.307030 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 9249
I20260810 06:49:40.307144 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:40.307355 16291 external_mini_cluster.cc:1428] Setting key 97c03c732dfdb8ab6930789833383c7a
I20260810 06:49:40.308636 9328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3: Bootstrap starting.
I20260810 06:49:40.309376 9328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:40.309690 9328 log.cc:824] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:40.310515 9328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3: No bootstrap required, opened a new log
I20260810 06:49:40.312456 9328 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec0aa7636ebe417b8f6fb389313bfff3" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43799 } }
I20260810 06:49:40.312645 9328 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:40.312711 9328 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec0aa7636ebe417b8f6fb389313bfff3, State: Initialized, Role: FOLLOWER
I20260810 06:49:40.312883 9328 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec0aa7636ebe417b8f6fb389313bfff3" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43799 } }
I20260810 06:49:40.312955 9328 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:40.312980 9328 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:40.313051 9328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:40.313652 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:40.308032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37711 (local address 127.15.232.254:43799)
0810 06:49:40.308238 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:40.308255 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:40.308284 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:49:40.308722 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:40.308729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:40.308818 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:40.308996 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:40.309244 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:40.310452 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:40.311359 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:40.311803 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:40.311952 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:40.312796 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:40.312802 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:40.312802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:40.313057 (+ 255us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:40.313101 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:40.313117 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:40.313218 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:40.313301 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 06:49:40.313379 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260810 06:49:40.313766 9328 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec0aa7636ebe417b8f6fb389313bfff3" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43799 } }
I20260810 06:49:40.313882 9328 leader_election.cc:302] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec0aa7636ebe417b8f6fb389313bfff3; no voters:
I20260810 06:49:40.314090 9328 leader_election.cc:288] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:40.314168 9333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:40.314374 9328 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:40.314697 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:40.314393 9333 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [term 1 LEADER]: Becoming Leader. State: Replica: ec0aa7636ebe417b8f6fb389313bfff3, State: Running, Role: LEADER
I20260810 06:49:40.315857 9333 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec0aa7636ebe417b8f6fb389313bfff3" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43799 } }
I20260810 06:49:40.316545 9335 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec0aa7636ebe417b8f6fb389313bfff3. Latest consensus state: current_term: 1 leader_uuid: "ec0aa7636ebe417b8f6fb389313bfff3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec0aa7636ebe417b8f6fb389313bfff3" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43799 } } }
I20260810 06:49:40.316656 9335 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:40.316556 9334 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec0aa7636ebe417b8f6fb389313bfff3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec0aa7636ebe417b8f6fb389313bfff3" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 43799 } } }
I20260810 06:49:40.316839 9334 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:40.317018 9356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:40.317577 9356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:40.319830 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:40.316119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54757 (local address 127.15.232.254:43799)
0810 06:49:40.316158 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:40.316159 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:40.316299 (+ 140us) server_negotiation.cc:407] Connection header received
0810 06:49:40.316400 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:40.316401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:40.316412 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:40.316442 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:40.316665 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:40.317471 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:40.318279 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:40.318607 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:40.318658 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:40.319561 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:40.319562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:40.319563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:40.319569 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:40.319584 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:40.319588 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:40.319616 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:40.319737 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 06:49:40.319764 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 06:49:40.320693 9356 catalog_manager.cc:1347] Generated new cluster ID: a21515d0166c4a0fbdd6ae78a2f1d003
I20260810 06:49:40.320740 9356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:40.339453 9356 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:40.340152 9356 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:40.345700 9356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec0aa7636ebe417b8f6fb389313bfff3: Generated new TSK 0
I20260810 06:49:40.345978 9356 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:40.359292 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 9249
2026-08-10T06:49:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:40 dist-test-slave-vxj2 krb5kdc[9363](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:40 dist-test-slave-vxj2 krb5kdc[9363](info): set up 2 sockets
Aug 10 06:49:40 dist-test-slave-vxj2 krb5kdc[9363](info): commencing operation
krb5kdc: starting...
W20260810 06:49:41.516988 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:41 dist-test-slave-vxj2 krb5kdc[9363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:49:41.648146 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:35031
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:42855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:35031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:41.740615 9376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:41.740859 9376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:41.740888 9376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:41.742338 9376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:41.742395 9376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:41.742411 9376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:41.742425 9376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:41.742439 9376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:41.744324 9376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:42855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:35031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:35031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:41.744633 9376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:41.744925 9376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:41.750705 9386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:41.750706 9382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:41.750706 9383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:41.751003 9376 server_base.cc:1034] running on GCE node
I20260810 06:49:41.751406 9376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:41.751713 9376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:41.752877 9376 hybrid_clock.cc:648] HybridClock initialized: now 1786344581752859 us; error 29 us; skew 500 ppm
Aug 10 06:49:41 dist-test-slave-vxj2 krb5kdc[9363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344581, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:41.755744 9376 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:49:41.756258 9376 webserver.cc:460] Webserver started at http://127.15.232.254:36885/ using document root <none> and password file <none>
I20260810 06:49:41.756552 9376 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:41.756603 9376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:41.756744 9376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:41.757984 9376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "51d0965f1a4543a0b32171f42302c680"
format_stamp: "Formatted at 2026-08-10 06:49:41 on dist-test-slave-vxj2"
server_key: "5ec801dc46626d3dadd109875afbec2c"
server_key_iv: "50670fd2493bdad4c7860df587399680"
server_key_version: "encryptionkey@0"
I20260810 06:49:41.758373 9376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "51d0965f1a4543a0b32171f42302c680"
format_stamp: "Formatted at 2026-08-10 06:49:41 on dist-test-slave-vxj2"
server_key: "5ec801dc46626d3dadd109875afbec2c"
server_key_iv: "50670fd2493bdad4c7860df587399680"
server_key_version: "encryptionkey@0"
I20260810 06:49:41.760057 9376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:49:41.761059 9394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:41.761345 9376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:41.761451 9376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "51d0965f1a4543a0b32171f42302c680"
format_stamp: "Formatted at 2026-08-10 06:49:41 on dist-test-slave-vxj2"
server_key: "5ec801dc46626d3dadd109875afbec2c"
server_key_iv: "50670fd2493bdad4c7860df587399680"
server_key_version: "encryptionkey@0"
I20260810 06:49:41.761534 9376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:41.765249 9376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:41.767390 9376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:41.767553 9376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:41.771781 9376 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:35031
I20260810 06:49:41.771817 9453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:35031 every 8 connection(s)
I20260810 06:49:41.772173 9376 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:41.773401 9454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:41.773972 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 9376
I20260810 06:49:41.774094 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:41.774298 16291 external_mini_cluster.cc:1428] Setting key 74e22bf66c48471787fb23ad70d1c606
I20260810 06:49:41.776682 9454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680: Bootstrap starting.
I20260810 06:49:41.776957 9457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:41.774881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52557 (local address 127.15.232.254:35031)
0810 06:49:41.775081 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:41.775089 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:41.775114 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:49:41.775510 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:41.775516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:41.775541 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:41.775660 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:41.776353 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:41.776357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:41.776357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:41.776361 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:41.776399 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:41.776410 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:41.776467 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:41.776562 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 06:49:41.776616 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
I20260810 06:49:41.777480 9454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:41.777837 9454 log.cc:824] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:41.777945 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:41.778658 9454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680: No bootstrap required, opened a new log
I20260810 06:49:41.780710 9457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:41.779683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36459 (local address 127.15.232.254:35031)
0810 06:49:41.779723 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:41.779726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:41.779795 (+ 69us) server_negotiation.cc:407] Connection header received
0810 06:49:41.779863 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:41.779864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:41.779868 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:41.779915 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:41.780452 (+ 537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:41.780453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:41.780453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:41.780457 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:41.780470 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:41.780473 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:41.780533 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:41.780609 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 06:49:41.780620 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:49:41.781520 9454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51d0965f1a4543a0b32171f42302c680" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35031 } }
I20260810 06:49:41.781685 9454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:41.781741 9454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51d0965f1a4543a0b32171f42302c680, State: Initialized, Role: FOLLOWER
I20260810 06:49:41.781874 9454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51d0965f1a4543a0b32171f42302c680" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35031 } }
I20260810 06:49:41.781980 9454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:41.782025 9454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:41.782078 9454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:41.782661 9454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51d0965f1a4543a0b32171f42302c680" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35031 } }
I20260810 06:49:41.782793 9454 leader_election.cc:302] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51d0965f1a4543a0b32171f42302c680; no voters:
I20260810 06:49:41.782976 9454 leader_election.cc:288] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:41.783092 9466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:41.783250 9466 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [term 1 LEADER]: Becoming Leader. State: Replica: 51d0965f1a4543a0b32171f42302c680, State: Running, Role: LEADER
I20260810 06:49:41.783264 9454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:41.783386 9466 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51d0965f1a4543a0b32171f42302c680" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35031 } }
I20260810 06:49:41.783813 9467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51d0965f1a4543a0b32171f42302c680" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51d0965f1a4543a0b32171f42302c680" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35031 } } }
I20260810 06:49:41.783820 9468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51d0965f1a4543a0b32171f42302c680. Latest consensus state: current_term: 1 leader_uuid: "51d0965f1a4543a0b32171f42302c680" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51d0965f1a4543a0b32171f42302c680" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 35031 } } }
I20260810 06:49:41.784005 9467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:41.784019 9468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:41.784477 9475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:41.784965 9475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:41.786662 9475 catalog_manager.cc:1347] Generated new cluster ID: 77fb55b3955145679cf22af0bdf9ef4a
I20260810 06:49:41.786785 9475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:41.797271 9475 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:41.797473 9475 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:41.808991 9475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51d0965f1a4543a0b32171f42302c680: Generated new TSK 0
I20260810 06:49:41.809314 9475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:41.814710 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 9376
2026-08-10T06:49:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1455 ms)
[----------] 18 tests from AuthTokenIssuingTest (26049 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:41 dist-test-slave-vxj2 krb5kdc[9490](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:41 dist-test-slave-vxj2 krb5kdc[9490](info): set up 2 sockets
Aug 10 06:49:41 dist-test-slave-vxj2 krb5kdc[9490](info): commencing operation
krb5kdc: starting...
W20260810 06:49:42.960285 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:49:43.088610 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:38101
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:35143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:38101,127.15.232.253:40767,127.15.232.252:37101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:43.173975 9505 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:49:43.174213 9505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:43.174232 9505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:43.174253 9505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:43.175443 9505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:43.175486 9505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:43.175500 9505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:43.175513 9505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:43.175526 9505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:43.177048 9505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:35143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:38101,127.15.232.253:40767,127.15.232.252:37101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:38101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:43.177304 9505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:43.177505 9505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:43.182921 9514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:43.182958 9513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:43.182958 9516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:43.183117 9505 server_base.cc:1034] running on GCE node
I20260810 06:49:43.183427 9505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:43.183732 9505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:43.184898 9505 hybrid_clock.cc:648] HybridClock initialized: now 1786344583184879 us; error 45 us; skew 500 ppm
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:43.187597 9505 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:49:43.188095 9505 webserver.cc:460] Webserver started at http://127.15.232.254:35623/ using document root <none> and password file <none>
I20260810 06:49:43.188333 9505 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:43.188410 9505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:43.188565 9505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:43.189749 9505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "0d6d583cbaed4c3e97f4c224a14e08ca"
format_stamp: "Formatted at 2026-08-10 06:49:43 on dist-test-slave-vxj2"
server_key: "8293768852470220ad29d1f93bb19d82"
server_key_iv: "91bbca0afc0393254d1dc93d3d29c21a"
server_key_version: "encryptionkey@0"
I20260810 06:49:43.190140 9505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "0d6d583cbaed4c3e97f4c224a14e08ca"
format_stamp: "Formatted at 2026-08-10 06:49:43 on dist-test-slave-vxj2"
server_key: "8293768852470220ad29d1f93bb19d82"
server_key_iv: "91bbca0afc0393254d1dc93d3d29c21a"
server_key_version: "encryptionkey@0"
I20260810 06:49:43.191566 9505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:49:43.192325 9524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:43.192516 9505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:49:43.192636 9505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "0d6d583cbaed4c3e97f4c224a14e08ca"
format_stamp: "Formatted at 2026-08-10 06:49:43 on dist-test-slave-vxj2"
server_key: "8293768852470220ad29d1f93bb19d82"
server_key_iv: "91bbca0afc0393254d1dc93d3d29c21a"
server_key_version: "encryptionkey@0"
I20260810 06:49:43.192716 9505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:43.203047 9505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:43.205353 9505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:43.205519 9505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:43.209574 9505 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:38101
I20260810 06:49:43.209635 9581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:38101 every 8 connection(s)
I20260810 06:49:43.210038 9505 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:43.210980 9582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:43.212606 9582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:43.214399 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 9505
I20260810 06:49:43.214519 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:43.214700 16291 external_mini_cluster.cc:1428] Setting key a8b95ca2786d280a8703fbd3119bb7a8
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.253@KRBTEST.COM; defaulting to no policy
I20260810 06:49:43.223961 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.213405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:38101 (local address 127.0.0.1:47356)
0810 06:49:43.213584 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.213624 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:43.214080 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:43.214430 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:43.214432 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:43.214759 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:43.214979 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.214985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.216578 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.216579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.217392 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.217395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.217924 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.217925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.217991 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.218632 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:43.218645 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:43.220383 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:43.222323 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.222330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.222333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.222569 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.222819 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.222820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.222820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.222910 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.223288 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:43.223294 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:43.223357 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 06:49:43.223472 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 06:49:43.223739 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":100,"thread_start_us":80,"threads_started":1}
I20260810 06:49:43.223961 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.213580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47356 (local address 127.15.232.254:38101)
0810 06:49:43.213712 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.213716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.213746 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:49:43.214266 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.214269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.214293 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.214414 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.215088 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.216439 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.217491 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.217819 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.217953 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.220483 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.220491 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.220491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.220498 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.222214 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.222648 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.222651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.222651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.222742 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.222984 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.222984 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.222985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.223212 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.223310 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.223507 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.223689 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
I20260810 06:49:43.224807 9582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.253@KRBTEST.COM" created.
I20260810 06:49:43.225274 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.225090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:40767 (local address 127.0.0.1:38190)
0810 06:49:43.225183 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.225242 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:40767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:49:43.225446 9530 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.253:40767: connect: Connection refused (error 111)
I20260810 06:49:43.225950 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.225842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:40767 (local address 127.0.0.1:38192)
0810 06:49:43.225889 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.225919 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:40767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 06:49:43.226073 9582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.253:40767: Network error: Client connection negotiation failed: client connection to 127.15.232.253:40767: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.253.keytab.
Entry for principal kudu/127.15.232.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.253.keytab.
I20260810 06:49:43.240684 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.253:40767
--webserver_interface=127.15.232.253
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.253.keytab
--principal=kudu/127.15.232.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:35143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:38101,127.15.232.253:40767,127.15.232.252:37101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:49:43.275360 9582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } attempt: 1
I20260810 06:49:43.275996 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.275823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:40767 (local address 127.0.0.1:38202)
0810 06:49:43.275908 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.275959 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:40767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 06:49:43.276441 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.276324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:40767 (local address 127.0.0.1:38206)
0810 06:49:43.276378 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.276418 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:40767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:49:43.276546 9582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.253:40767: Network error: Client connection negotiation failed: client connection to 127.15.232.253:40767: connect: Connection refused (error 111)
W20260810 06:49:43.326225 9590 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:49:43.326550 9590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:43.326581 9590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:43.326627 9590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:43.327921 9590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:43.328009 9590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:43.328037 9590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:43.328058 9590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:43.328086 9590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:43.329769 9590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:35143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:38101,127.15.232.253:40767,127.15.232.252:37101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.253.keytab
--principal=kudu/127.15.232.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.253:40767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.232.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:43.330078 9590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:43.330327 9590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:43.335798 9596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:43.335930 9599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:43.335824 9595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:43.335918 9590 server_base.cc:1034] running on GCE node
I20260810 06:49:43.336316 9590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:43.336608 9590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:43.337766 9590 hybrid_clock.cc:648] HybridClock initialized: now 1786344583337744 us; error 35 us; skew 500 ppm
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:43.340677 9590 init.cc:377] Logged in from keytab as kudu/127.15.232.253@KRBTEST.COM (short username kudu)
I20260810 06:49:43.341246 9590 webserver.cc:460] Webserver started at http://127.15.232.253:40611/ using document root <none> and password file <none>
I20260810 06:49:43.341488 9590 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:43.341537 9590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:43.341675 9590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:43.342818 9590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/data/instance:
uuid: "a6cb09b833e74951817f81ad02e2aee7"
format_stamp: "Formatted at 2026-08-10 06:49:43 on dist-test-slave-vxj2"
server_key: "bbcb4ab3b9983ac987b5da2c791838d9"
server_key_iv: "7af893b4db7c40d2cc06e8851828f0df"
server_key_version: "encryptionkey@0"
I20260810 06:49:43.343200 9590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/wal/instance:
uuid: "a6cb09b833e74951817f81ad02e2aee7"
format_stamp: "Formatted at 2026-08-10 06:49:43 on dist-test-slave-vxj2"
server_key: "bbcb4ab3b9983ac987b5da2c791838d9"
server_key_iv: "7af893b4db7c40d2cc06e8851828f0df"
server_key_version: "encryptionkey@0"
I20260810 06:49:43.344636 9590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:49:43.344692 9582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } attempt: 2
I20260810 06:49:43.345263 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.345056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:40767 (local address 127.0.0.1:38208)
0810 06:49:43.345124 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.345210 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:40767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 06:49:43.345676 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.345531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:40767 (local address 127.0.0.1:38216)
0810 06:49:43.345603 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.345641 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:40767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:49:43.345793 9582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.253:40767: Network error: Client connection negotiation failed: client connection to 127.15.232.253:40767: connect: Connection refused (error 111)
I20260810 06:49:43.345886 9608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:43.346078 9590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:49:43.346184 9590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/wal
uuid: "a6cb09b833e74951817f81ad02e2aee7"
format_stamp: "Formatted at 2026-08-10 06:49:43 on dist-test-slave-vxj2"
server_key: "bbcb4ab3b9983ac987b5da2c791838d9"
server_key_iv: "7af893b4db7c40d2cc06e8851828f0df"
server_key_version: "encryptionkey@0"
I20260810 06:49:43.346263 9590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:43.360523 9590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:43.362950 9590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:43.363134 9590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:43.367559 9590 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.253:40767
I20260810 06:49:43.367605 9666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.253:40767 every 8 connection(s)
I20260810 06:49:43.368016 9590 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
I20260810 06:49:43.368993 9668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:43.370767 9668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:43.376879 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 9590
I20260810 06:49:43.376991 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-1/wal/instance
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.253@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:43.377297 16291 external_mini_cluster.cc:1428] Setting key 91e1609993b210e3ad9ff006533212f3
I20260810 06:49:43.381671 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.371570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47368 (local address 127.15.232.254:38101)
0810 06:49:43.371647 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.371649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.371889 (+ 240us) server_negotiation.cc:407] Connection header received
0810 06:49:43.372346 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.372347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.372361 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.372436 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.373275 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.373795 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.374940 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.375183 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.375257 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.377800 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.377801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.377802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.377810 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.379523 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.380021 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.380022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.380023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.380138 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.380554 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.380556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.380556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.380792 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.380889 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.381416 (+ 527us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.381533 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:49:43.382359 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.371482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:38101 (local address 127.0.0.1:47368)
0810 06:49:43.371682 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.371724 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:43.372216 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:43.372550 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:43.372554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:43.372875 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:43.373103 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.373108 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.373909 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.373910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.374829 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.374831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.375300 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.375302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.375422 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.375975 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:43.375984 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:43.377680 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:43.379642 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.379652 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.379655 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.379927 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.380213 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.380214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.380214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.380457 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.380929 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:43.380953 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:43.381205 (+ 252us) client_negotiation.cc:769] Sending connection context
0810 06:49:43.381855 (+ 650us) client_negotiation.cc:240] Negotiation successful
0810 06:49:43.382081 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260810 06:49:43.386922 9668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.232.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.252@KRBTEST.COM" created.
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.253@KRBTEST.COM for kudu/127.15.232.253@KRBTEST.COM
I20260810 06:49:43.397375 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.387370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:40767 (local address 127.0.0.1:38228)
0810 06:49:43.387435 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.387448 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:43.387491 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:43.387866 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:43.387867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:43.388075 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:43.388184 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.388187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.389728 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.389730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.390316 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.390319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.390939 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.390940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.391080 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.391633 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:43.391639 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:43.393417 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:43.395836 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.395838 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.395838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.396131 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.396415 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.396416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.396417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.396497 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.396847 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:43.396848 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:43.396916 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 06:49:43.397000 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 06:49:43.397133 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 06:49:43.397379 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.387464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.15.232.253:40767)
0810 06:49:43.387658 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.387664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.387689 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:49:43.387736 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.387739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.387756 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.388287 (+ 531us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.388377 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.389635 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.390463 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.390848 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.390944 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.393530 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.393544 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.393545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.393555 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.395727 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.396232 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.396236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.396237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.396334 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.396597 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.396598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.396599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.396772 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.396869 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.397040 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.397252 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":162,"thread_start_us":117,"threads_started":1}
I20260810 06:49:43.398207 9668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:43.398617 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.398476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50014)
0810 06:49:43.398530 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.398587 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 06:49:43.398828 9611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
I20260810 06:49:43.399344 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.399228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50028)
0810 06:49:43.399282 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.399315 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:49:43.399469 9668 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:37101: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.252.keytab.
Entry for principal kudu/127.15.232.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.252.keytab.
I20260810 06:49:43.408536 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.252:37101
--webserver_interface=127.15.232.252
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.252.keytab
--principal=kudu/127.15.232.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:35143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:38101,127.15.232.253:40767,127.15.232.252:37101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:49:43.437013 9582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } attempt: 3
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.253@KRBTEST.COM
I20260810 06:49:43.446750 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.437721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38236 (local address 127.15.232.253:40767)
0810 06:49:43.437805 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.437808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.437823 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:49:43.437859 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.437860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.437875 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.437943 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.438303 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.439102 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.440157 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.440534 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.440619 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.443197 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.443199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.443200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.443220 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.445328 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.445804 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.445806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.445806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.445879 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.446117 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.446118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.446118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.446289 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.446355 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.446593 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.446679 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:49:43.446751 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.437597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:40767 (local address 127.0.0.1:38236)
0810 06:49:43.437707 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.437723 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:43.437802 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:43.437966 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:43.437967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:43.438139 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:43.438212 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.438214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.439270 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.439271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.440048 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.440053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.440645 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.440647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.440741 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.441402 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:43.441408 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:43.443038 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:43.445439 (+ 2401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.445440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.445441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.445695 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.445968 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.445970 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.445970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.446047 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.446363 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:43.446364 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:43.446432 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 06:49:43.446522 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 06:49:43.446640 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 06:49:43.447332 9582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:43.447741 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.447614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50036)
0810 06:49:43.447667 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.447712 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 06:49:43.448073 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.447978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50052)
0810 06:49:43.448026 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.448055 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:49:43.448199 9582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:37101: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
I20260810 06:49:43.448705 9668 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } attempt: 1
I20260810 06:49:43.449138 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.449006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50058)
0810 06:49:43.449075 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.449111 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 06:49:43.449692 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.449567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50060)
0810 06:49:43.449621 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.449656 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 06:49:43.449819 9668 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:37101: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
I20260810 06:49:43.479398 9582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } attempt: 1
I20260810 06:49:43.480037 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.479843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50068)
0810 06:49:43.479924 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.479986 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 06:49:43.480584 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.480444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50070)
0810 06:49:43.480508 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.480560 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:49:43.480741 9582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:37101: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
W20260810 06:49:43.499159 9676 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:49:43.499445 9676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:43.499488 9676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:43.499533 9676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:43.500779 9676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:43.500991 9676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:43.501088 9676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:43.501127 9676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:43.501197 9676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:43.503274 9676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:35143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:38101,127.15.232.253:40767,127.15.232.252:37101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.252.keytab
--principal=kudu/127.15.232.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.252:37101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.232.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:43.503640 9676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:43.503929 9676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:43.510075 9687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:43.510124 9684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:43.510082 9685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:43.510572 9676 server_base.cc:1034] running on GCE node
I20260810 06:49:43.510926 9676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:43.511267 9676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:43.512432 9676 hybrid_clock.cc:648] HybridClock initialized: now 1786344583512403 us; error 42 us; skew 500 ppm
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:43.515786 9676 init.cc:377] Logged in from keytab as kudu/127.15.232.252@KRBTEST.COM (short username kudu)
I20260810 06:49:43.516374 9676 webserver.cc:460] Webserver started at http://127.15.232.252:43531/ using document root <none> and password file <none>
I20260810 06:49:43.516654 9676 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:43.516746 9676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:43.516943 9676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:43.517951 9668 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } attempt: 2
I20260810 06:49:43.518356 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.518227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50086)
0810 06:49:43.518288 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.518327 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:49:43.518513 9676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/data/instance:
uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e"
format_stamp: "Formatted at 2026-08-10 06:49:43 on dist-test-slave-vxj2"
server_key: "1978944ef73b61e18fc95246daa3ec9e"
server_key_iv: "7dc7502eabe437c9642f721bb2a5656c"
server_key_version: "encryptionkey@0"
I20260810 06:49:43.518702 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.518601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50096)
0810 06:49:43.518645 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.518671 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 06:49:43.518800 9668 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:37101: Network error: Client connection negotiation failed: client connection to 127.15.232.252:37101: connect: Connection refused (error 111)
I20260810 06:49:43.519048 9676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/wal/instance:
uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e"
format_stamp: "Formatted at 2026-08-10 06:49:43 on dist-test-slave-vxj2"
server_key: "1978944ef73b61e18fc95246daa3ec9e"
server_key_iv: "7dc7502eabe437c9642f721bb2a5656c"
server_key_version: "encryptionkey@0"
I20260810 06:49:43.520864 9676 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260810 06:49:43.521943 9696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:43.522181 9676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:49:43.522300 9676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/wal
uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e"
format_stamp: "Formatted at 2026-08-10 06:49:43 on dist-test-slave-vxj2"
server_key: "1978944ef73b61e18fc95246daa3ec9e"
server_key_iv: "7dc7502eabe437c9642f721bb2a5656c"
server_key_version: "encryptionkey@0"
I20260810 06:49:43.522392 9676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:43.543915 9676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:43.546877 9676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:43.547050 9676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:43.552104 9676 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.252:37101
I20260810 06:49:43.552181 9756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.252:37101 every 8 connection(s)
I20260810 06:49:43.552579 9676 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
I20260810 06:49:43.553625 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:43.554695 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 9676
I20260810 06:49:43.554873 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344303981681-16291-0/minicluster-data/master-2/wal/instance
I20260810 06:49:43.555080 16291 external_mini_cluster.cc:1428] Setting key 3352be64dd114bcba5e3786cf089c6b4
I20260810 06:49:43.555606 9757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:43.555910 9582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } attempt: 2
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.252@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.252@KRBTEST.COM
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.252@KRBTEST.COM
I20260810 06:49:43.565856 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.556560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47374 (local address 127.15.232.254:38101)
0810 06:49:43.556608 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.556610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.556618 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:49:43.556827 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.556828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.556839 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.556895 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.557603 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.558374 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.559335 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.559542 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.559620 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.562227 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.562229 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.562229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.562237 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.563853 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.564549 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.564550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.564551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.564619 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.564952 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.564953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.564954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.565217 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.565300 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.565653 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.565750 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:49:43.566449 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.556203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50108)
0810 06:49:43.556270 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.556280 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:43.556319 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:43.556964 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:43.556965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:43.557137 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:43.557218 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.557220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.558459 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.558460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.559254 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.559256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.559886 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.559887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.559947 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.560575 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:43.560585 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:43.562861 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:43.565170 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.565171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.565171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.565470 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.565723 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.565724 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.565725 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.565796 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.566171 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:43.566172 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:43.566223 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:49:43.566276 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 06:49:43.566366 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 06:49:43.566534 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.556305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:38101 (local address 127.0.0.1:47374)
0810 06:49:43.556455 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.556496 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:43.556717 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:43.556914 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:43.556917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:43.557333 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:43.557510 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.557516 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.558477 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.558478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.559241 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.559244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.559684 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.559685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.559817 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.560430 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:43.560442 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:43.562124 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:43.563942 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.563951 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.563953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.564460 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.564695 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.564696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.564696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.564854 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.565336 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:43.565343 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:43.565474 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 06:49:43.566008 (+ 534us) client_negotiation.cc:240] Negotiation successful
0810 06:49:43.566252 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":57,"threads_started":1}
I20260810 06:49:43.567082 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.556237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50108 (local address 127.15.232.252:37101)
0810 06:49:43.556380 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.556382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.556392 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:49:43.556717 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.556718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.556742 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.556832 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.557318 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.558352 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.559351 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.559803 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.559885 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.562963 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.562973 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.562974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.562982 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.565064 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.565565 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.565569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.565569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.565645 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.565883 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.565884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.565884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.566079 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.566655 (+ 576us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.566792 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.566966 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260810 06:49:43.567102 9757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:43.568099 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.555702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.15.232.252:37101)
0810 06:49:43.556014 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.556022 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.556071 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:49:43.556700 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.556708 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.556740 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.556843 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.557744 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.558483 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.559606 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.559893 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.559958 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.563500 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.563502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.563502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.563510 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.566194 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.566597 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.566598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.566599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.566668 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.566861 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.566862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.566863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.566987 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.567126 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.567882 (+ 756us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.567979 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":406,"server-negotiator.queue_time_us":259,"thread_start_us":161,"threads_started":1}
I20260810 06:49:43.569599 9582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca: Bootstrap starting.
I20260810 06:49:43.570431 9582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:43.570775 9582 log.cc:824] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:43.571727 9582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca: No bootstrap required, opened a new log
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.252@KRBTEST.COM for kudu/127.15.232.253@KRBTEST.COM
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:43.574193 9582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } }
I20260810 06:49:43.574379 9582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d6d583cbaed4c3e97f4c224a14e08ca, State: Initialized, Role: FOLLOWER
I20260810 06:49:43.574477 9582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } }
I20260810 06:49:43.574887 9582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:43.574966 9767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } } }
I20260810 06:49:43.575067 9767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:43.575773 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.567488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.15.232.253:40767)
0810 06:49:43.567533 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.567535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.567577 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:49:43.567659 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.567660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.567671 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.567722 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.568104 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.568851 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.569866 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.570129 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.570211 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.572346 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.572349 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.572349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.572359 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.574142 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.574669 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.574671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.574671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.574741 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.575037 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.575038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.575038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.575195 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.575243 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.575551 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.575658 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:49:43.576329 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.567423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:40767 (local address 127.0.0.1:38242)
0810 06:49:43.567531 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.567542 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:43.567592 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:43.567772 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:43.567773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:43.567956 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:43.568004 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.568006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.568947 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.568949 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.569772 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.569775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.570206 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.570207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.570272 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.570730 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:43.570735 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:43.572244 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:43.574243 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.574244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.574245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.574479 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.574832 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.574833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.574834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.574905 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.575352 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:43.575353 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:43.575410 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 06:49:43.576061 (+ 651us) client_negotiation.cc:240] Negotiation successful
0810 06:49:43.576202 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 06:49:43.577114 9757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } has no permanent_uuid. Determining permanent_uuid...
W20260810 06:49:43.576925 9779 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 06:49:43.577688 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.569215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47386 (local address 127.15.232.254:38101)
0810 06:49:43.569255 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.569257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.569264 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:49:43.569283 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.569283 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.569292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.569363 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.569921 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.570640 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.571532 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.571840 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.571930 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.574384 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.574386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.574386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.574394 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.576170 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.576589 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.576590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.576590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.576669 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.576891 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.576892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.576892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.577030 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.577189 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.577464 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.577551 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 06:49:43.577560 9779 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:49:43.578372 9505 master_runner.cc:186] Error getting master registration for 127.15.232.253:40767: OK, instance_id { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" instance_seqno: 1786344583366218 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:49:43.580456 9505 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.252@KRBTEST.COM for kudu/127.15.232.252@KRBTEST.COM
I20260810 06:49:43.586387 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.577406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50114)
0810 06:49:43.577649 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.577661 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:43.577917 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:43.578269 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:43.578270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:43.578428 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:43.578470 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.578472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.579371 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.579372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.580175 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.580178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.580667 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.580668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.580729 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.581132 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:43.581137 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:43.582690 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:43.584728 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.584729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.584729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.585294 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.585507 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.585508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.585508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.585583 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.585986 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:43.585987 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:43.586038 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:49:43.586118 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:49:43.586242 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224}
I20260810 06:49:43.586760 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.577490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.15.232.252:37101)
0810 06:49:43.577525 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.577527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.577699 (+ 172us) server_negotiation.cc:407] Connection header received
0810 06:49:43.577985 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.577986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.577996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.578027 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.578559 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.579271 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.580276 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.580584 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.580659 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.582787 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.582788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.582788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.582796 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.584618 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.585366 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.585367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.585368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.585426 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.585658 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.585659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.585659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.585901 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.585983 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.586187 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.586310 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":16}
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.253@KRBTEST.COM
I20260810 06:49:43.588976 9757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e: Bootstrap starting.
I20260810 06:49:43.589885 9757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:43.590212 9757 log.cc:824] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:43.590430 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.578320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38252 (local address 127.15.232.253:40767)
0810 06:49:43.578354 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.578357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.581352 (+ 2995us) server_negotiation.cc:407] Connection header received
0810 06:49:43.581422 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.581423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.581434 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.581482 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.581984 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.582486 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.583393 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.583714 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.583760 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.587611 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.587612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.587613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.587620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.589218 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.589596 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.589598 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.589598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.589655 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.589885 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.589886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.589886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.590024 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.590209 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.590245 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.590322 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 06:49:43.591094 9757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e: No bootstrap required, opened a new log
I20260810 06:49:43.591921 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:43.592895 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47386
I20260810 06:49:43.593066 9554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 06:49:43.593122 9554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:43.593237 9757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } }
I20260810 06:49:43.593421 9757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8aab5dcecf433a9b0437dbb6c7e92e, State: Initialized, Role: FOLLOWER
I20260810 06:49:43.593513 9757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } }
I20260810 06:49:43.593868 9757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:43.593827 9782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } } }
I20260810 06:49:43.594033 9782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:43.594240 9554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } }
I20260810 06:49:43.594527 9554 leader_election.cc:288] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [CANDIDATE]: Term 1 election: Requested vote from peers a6cb09b833e74951817f81ad02e2aee7 (127.15.232.253:40767), 3e8aab5dcecf433a9b0437dbb6c7e92e (127.15.232.252:37101)
I20260810 06:49:43.594846 9728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e"
I20260810 06:49:43.595002 9728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:43.596146 9728 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d6d583cbaed4c3e97f4c224a14e08ca in term 1.
I20260810 06:49:43.596487 9526 leader_election.cc:302] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d6d583cbaed4c3e97f4c224a14e08ca, 3e8aab5dcecf433a9b0437dbb6c7e92e; no voters:
I20260810 06:49:43.596644 9767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:43.596793 9767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [term 1 LEADER]: Becoming Leader. State: Replica: 0d6d583cbaed4c3e97f4c224a14e08ca, State: Running, Role: LEADER
I20260810 06:49:43.596881 9767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } }
I20260810 06:49:43.597453 9767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d6d583cbaed4c3e97f4c224a14e08ca. Latest consensus state: current_term: 1 leader_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } } }
I20260810 06:49:43.597537 9767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [sys.catalog]: This master's current role is: LEADER
W20260810 06:49:43.598389 9796 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:49:43.598450 9796 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:49:43.599561 9676 master_runner.cc:186] Error getting master registration for 127.15.232.253:40767: OK, instance_id { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" instance_seqno: 1786344583366218 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:49:43.601280 9676 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:49:43.601476 9806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:43.601698 9806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:43.603420 9728 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [term 1 FOLLOWER]: Refusing update from remote peer 0d6d583cbaed4c3e97f4c224a14e08ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:43.603798 9797 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:43.605135 9782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d6d583cbaed4c3e97f4c224a14e08ca. Latest consensus state: current_term: 1 leader_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } } }
I20260810 06:49:43.605276 9782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:43.606380 9767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } } }
I20260810 06:49:43.606495 9767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:43.606642 9782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } } }
I20260810 06:49:43.606725 9782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:43.607025 9809 mvcc.cc:205] Tried to move back new op lower bound from 7316867414436691968 to 7316867414413877248. Current Snapshot: MvccSnapshot[applied={T|T < 7316867414436691968}]
I20260810 06:49:43.607136 9810 mvcc.cc:205] Tried to move back new op lower bound from 7316867414436691968 to 7316867414413877248. Current Snapshot: MvccSnapshot[applied={T|T < 7316867414436691968}]
I20260810 06:49:43.608033 9806 catalog_manager.cc:1347] Generated new cluster ID: b904ec3ad2514ad3a58f3474c59fcbde
I20260810 06:49:43.608099 9806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:43.609918 9668 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } attempt: 3
Aug 10 06:49:43 dist-test-slave-vxj2 krb5kdc[9490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.253@KRBTEST.COM for kudu/127.15.232.252@KRBTEST.COM
I20260810 06:49:43.619321 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.610198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:37101 (local address 127.0.0.1:50134)
0810 06:49:43.610280 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.610291 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:43.610327 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:43.611095 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:43.611096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:43.611269 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:43.611312 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.611313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.612260 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.612261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.613037 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.613039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.613513 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.613514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.613567 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.614058 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:43.614078 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:43.615641 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:43.617874 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.617875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.617876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.618117 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.618360 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.618361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.618361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.618433 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.618981 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:43.618982 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:43.619032 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 06:49:43.619098 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 06:49:43.619200 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 06:49:43.619724 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.610392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50134 (local address 127.15.232.252:37101)
0810 06:49:43.610965 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.610967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.610975 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:49:43.610998 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.610998 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.611010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.611063 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.611367 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.612171 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.613098 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.613444 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.613514 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.615744 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.615746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.615746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.615754 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.617769 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.618215 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.618216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.618216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.618292 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.618499 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.618500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.618500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.618623 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.618681 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.619545 (+ 864us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.619632 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545}
I20260810 06:49:43.621831 9668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7: Bootstrap starting.
I20260810 06:49:43.622694 9668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:43.623059 9668 log.cc:824] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:43.628491 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.597956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50118 (local address 127.15.232.252:37101)
0810 06:49:43.598000 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.598002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.614683 (+ 16681us) server_negotiation.cc:407] Connection header received
0810 06:49:43.614772 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.614773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.614784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.614812 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.616302 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.617036 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.618098 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.618384 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.618425 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.625012 (+ 6587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.625014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.625014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.625022 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.626838 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.627448 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.627450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.627450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.627502 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.627781 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.627781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.627782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.627929 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.628002 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.628317 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.628399 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:49:43.629962 9668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7: No bootstrap required, opened a new log
I20260810 06:49:43.631042 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.597858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47400 (local address 127.15.232.254:38101)
0810 06:49:43.597900 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.597902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.613684 (+ 15782us) server_negotiation.cc:407] Connection header received
0810 06:49:43.613788 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.613789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.613802 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.613833 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.616287 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.617077 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.618996 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.619280 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.619321 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.622892 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.622894 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.622894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.622905 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.624738 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.629896 (+ 5158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.629897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.629898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.629963 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.630323 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.630324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.630324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.630467 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.630544 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.630853 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.630946 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:49:43.632287 9668 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } }
I20260810 06:49:43.632460 9668 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6cb09b833e74951817f81ad02e2aee7, State: Initialized, Role: FOLLOWER
I20260810 06:49:43.632548 9668 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } }
I20260810 06:49:43.632913 9668 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:43.633119 9640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a6cb09b833e74951817f81ad02e2aee7"
I20260810 06:49:43.633605 9639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:43.634701 9639 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [term 1 FOLLOWER]: Refusing update from remote peer 0d6d583cbaed4c3e97f4c224a14e08ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:43.635113 9797 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:43.636267 9640 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d6d583cbaed4c3e97f4c224a14e08ca in term 1.
I20260810 06:49:43.637068 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.597734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.15.232.253:40767)
0810 06:49:43.598044 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.598047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.601356 (+ 3309us) server_negotiation.cc:407] Connection header received
0810 06:49:43.602624 (+ 1268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.602625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.602640 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.602714 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.610098 (+ 7384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.610854 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.615542 (+ 4688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.615866 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.615920 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.632294 (+ 16374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.632296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.632296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.632305 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.635655 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.636085 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.636087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.636087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.636147 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.636408 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.636409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.636409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.636576 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.636671 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.636886 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.636983 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282}
I20260810 06:49:43.637362 9815 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } } }
I20260810 06:49:43.637547 9815 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:43.637651 9815 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d6d583cbaed4c3e97f4c224a14e08ca. Latest consensus state: current_term: 1 leader_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } } }
I20260810 06:49:43.637753 9815 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:43.637836 9815 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } } }
I20260810 06:49:43.637928 9806 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:43.638013 9815 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:43.638677 9806 catalog_manager.cc:1504] Loading token signing keys...
W20260810 06:49:43.641925 9827 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:49:43.641991 9827 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:49:43.643013 9830 mvcc.cc:205] Tried to move back new op lower bound from 7316867414556819456 to 7316867414413877248. Current Snapshot: MvccSnapshot[applied={T|T < 7316867414436691968}]
I20260810 06:49:43.648244 9797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6d583cbaed4c3e97f4c224a14e08ca" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 38101 } } peers { permanent_uuid: "a6cb09b833e74951817f81ad02e2aee7" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 40767 } } peers { permanent_uuid: "3e8aab5dcecf433a9b0437dbb6c7e92e" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 37101 } } }
I20260810 06:49:43.648368 9797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:43.651108 9806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d6d583cbaed4c3e97f4c224a14e08ca: Generated new TSK 0
I20260810 06:49:43.651327 9806 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:49:43.677723 9672 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:43.675875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38268 (local address 127.15.232.253:40767)
0810 06:49:43.675925 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.675928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.676946 (+ 1018us) server_negotiation.cc:407] Connection header received
0810 06:49:43.677055 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.677056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.677068 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.677099 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.677644 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38268: BlockingRecv error: recv got EOF from 127.0.0.1:38268 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 06:49:43.678704 9585 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:43.676358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47402 (local address 127.15.232.254:38101)
0810 06:49:43.676412 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.676414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.678142 (+ 1728us) server_negotiation.cc:407] Connection header received
0810 06:49:43.678243 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.678244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.678256 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.678286 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.678634 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47402: BlockingRecv error: recv got EOF from 127.0.0.1:47402 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 06:49:43.679605 9760 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:43.676691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50150 (local address 127.15.232.252:37101)
0810 06:49:43.676734 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.676736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.679042 (+ 2306us) server_negotiation.cc:407] Connection header received
0810 06:49:43.679170 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.679172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.679183 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.679213 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.679527 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50150: BlockingRecv error: recv got EOF from 127.0.0.1:50150 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 06:49:43.686034 9672 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:43.681824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.15.232.253:40767)
0810 06:49:43.681870 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.681872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.685393 (+ 3521us) server_negotiation.cc:407] Connection header received
0810 06:49:43.685533 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.685534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.685556 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.685587 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.685977 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38278: BlockingRecv error: recv got EOF from 127.0.0.1:38278 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 06:49:43.689553 9672 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:43.688503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38294 (local address 127.15.232.253:40767)
0810 06:49:43.688543 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.688545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.688841 (+ 296us) server_negotiation.cc:407] Connection header received
0810 06:49:43.688953 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.688954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.688966 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.688996 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.689496 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38294: BlockingRecv error: recv got EOF from 127.0.0.1:38294 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 06:49:43.694028 9672 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:43.693074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.15.232.253:40767)
0810 06:49:43.693113 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.693115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.693373 (+ 258us) server_negotiation.cc:407] Connection header received
0810 06:49:43.693499 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.693500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.693512 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.693543 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.693964 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38308: BlockingRecv error: recv got EOF from 127.0.0.1:38308 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 06:49:43.700520 9672 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:43.699499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38312 (local address 127.15.232.253:40767)
0810 06:49:43.699543 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.699546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.699855 (+ 309us) server_negotiation.cc:407] Connection header received
0810 06:49:43.699997 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.699998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.700010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.700048 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.700463 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38312: BlockingRecv error: recv got EOF from 127.0.0.1:38312 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 06:49:43.711185 9672 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:43.710135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38322 (local address 127.15.232.253:40767)
0810 06:49:43.710184 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.710187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.710525 (+ 338us) server_negotiation.cc:407] Connection header received
0810 06:49:43.710641 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.710642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.710655 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.710687 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.711109 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38322: BlockingRecv error: recv got EOF from 127.0.0.1:38322 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 06:49:43.729821 9672 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:43.728753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38338 (local address 127.15.232.253:40767)
0810 06:49:43.728803 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.728805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.729127 (+ 322us) server_negotiation.cc:407] Connection header received
0810 06:49:43.729271 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.729272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.729284 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.729315 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.729749 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38338: BlockingRecv error: recv got EOF from 127.0.0.1:38338 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 06:49:43.764490 9672 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:43.763530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38346 (local address 127.15.232.253:40767)
0810 06:49:43.763580 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.763582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.763870 (+ 288us) server_negotiation.cc:407] Connection header received
0810 06:49:43.763973 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.763974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.763986 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.764015 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.764421 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38346: BlockingRecv error: recv got EOF from 127.0.0.1:38346 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 06:49:43.830973 9672 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:43.830146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38358 (local address 127.15.232.253:40767)
0810 06:49:43.830204 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.830206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.830353 (+ 147us) server_negotiation.cc:407] Connection header received
0810 06:49:43.830394 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.830394 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.830406 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.830464 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.830915 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38358: BlockingRecv error: recv got EOF from 127.0.0.1:38358 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260810 06:49:43.961253 9672 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:43.960348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38370 (local address 127.15.232.253:40767)
0810 06:49:43.960419 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.960421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.960492 (+ 71us) server_negotiation.cc:407] Connection header received
0810 06:49:43.960548 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.960549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.960559 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.960638 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.961118 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38370: BlockingRecv error: recv got EOF from 127.0.0.1:38370 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 06:49:44.219527 9672 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:44.218708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.15.232.253:40767)
0810 06:49:44.218784 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:44.218786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:44.218796 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:49:44.218932 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:44.218934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:44.218947 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:44.219023 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:44.219456 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38374: BlockingRecv error: recv got EOF from 127.0.0.1:38374 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:49:44.598955 9796 catalog_manager.cc:1259] Loaded cluster ID: b904ec3ad2514ad3a58f3474c59fcbde
I20260810 06:49:44.599076 9796 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e: loading cluster ID for follower catalog manager: success
I20260810 06:49:44.599954 9796 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e: acquiring CA information for follower catalog manager: success
I20260810 06:49:44.600283 9796 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3e8aab5dcecf433a9b0437dbb6c7e92e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:49:44.642530 9827 catalog_manager.cc:1259] Loaded cluster ID: b904ec3ad2514ad3a58f3474c59fcbde
I20260810 06:49:44.642678 9827 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7: loading cluster ID for follower catalog manager: success
I20260810 06:49:44.643519 9827 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7: acquiring CA information for follower catalog manager: success
I20260810 06:49:44.643888 9827 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a6cb09b833e74951817f81ad02e2aee7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:49:44.735786 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:44.732849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38822 (local address 127.15.232.253:40767)
0810 06:49:44.733017 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:44.733020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:44.733048 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:49:44.733079 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:44.733081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:44.733205 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:44.733324 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:44.733494 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.734009 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.734868 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.735121 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.735233 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:44.735416 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:44.735539 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:44.735633 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:44.735656 (+ 23us) server_negotiation.cc:299] Negotiation successful
0810 06:49:44.735680 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260810 06:49:44.740420 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:44.737936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55940 (local address 127.15.232.252:37101)
0810 06:49:44.738055 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:44.738057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:44.738063 (+ 6us) server_negotiation.cc:407] Connection header received
0810 06:49:44.738101 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:44.738101 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:44.738139 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:44.738233 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:44.738339 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.738827 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.739595 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.739802 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.739889 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:44.740074 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:44.740181 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:44.740275 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:44.740300 (+ 25us) server_negotiation.cc:299] Negotiation successful
0810 06:49:44.740321 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":53,"threads_started":1}
I20260810 06:49:44.745513 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:44.742240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38826 (local address 127.15.232.253:40767)
0810 06:49:44.742360 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:44.742362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:44.742562 (+ 200us) server_negotiation.cc:407] Connection header received
0810 06:49:44.742685 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:44.742686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:44.742697 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:44.742747 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:44.742934 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.743496 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.744607 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.744917 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.744975 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:44.745221 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:44.745254 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:44.745308 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:44.745383 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 06:49:44.745409 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 06:49:44.745615 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:44.742552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55948 (local address 127.15.232.252:37101)
0810 06:49:44.742593 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:44.742595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:44.742868 (+ 273us) server_negotiation.cc:407] Connection header received
0810 06:49:44.742968 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:44.742969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:44.742980 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:44.743036 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:44.743143 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.743834 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.744571 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.744865 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.744933 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:44.745083 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:44.745119 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:44.745206 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:44.745428 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 06:49:44.745455 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:49:44.745641 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:44.742241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35058 (local address 127.15.232.254:38101)
0810 06:49:44.742524 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:44.742527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:44.742537 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:49:44.742585 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:44.742586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:44.742642 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:44.742736 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:44.742855 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.743669 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.744652 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.744882 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.744962 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:44.745190 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:44.745305 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:44.745364 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:44.745532 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 06:49:44.745563 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":68,"threads_started":1}
I20260810 06:49:44.747915 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 9505
I20260810 06:49:44.757330 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 9590
I20260810 06:49:44.765596 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 9676
2026-08-10T06:49:44Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2947 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:44 dist-test-slave-vxj2 krb5kdc[9932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:44 dist-test-slave-vxj2 krb5kdc[9932](info): set up 2 sockets
Aug 10 06:49:44 dist-test-slave-vxj2 krb5kdc[9932](info): commencing operation
krb5kdc: starting...
W20260810 06:49:45.942637 16291 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/test-user.keytab.
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
Entry for principal kudu/127.15.232.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab.
I20260810 06:49:46.076563 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:34119
--webserver_interface=127.15.232.254
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:41859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:34119,127.15.232.253:43719,127.15.232.252:38889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:46.166867 9947 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:49:46.167140 9947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:46.167172 9947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:46.167218 9947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:46.168454 9947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:46.168527 9947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:46.168550 9947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:46.168571 9947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:46.168598 9947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:46.170233 9947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:41859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:34119,127.15.232.253:43719,127.15.232.252:38889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.254.keytab
--principal=kudu/127.15.232.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.254:34119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:46.170552 9947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:46.170799 9947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:46.175938 9952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:46.176028 9953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:46.175993 9955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:46.176173 9947 server_base.cc:1034] running on GCE node
I20260810 06:49:46.176445 9947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:46.176749 9947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:46.177946 9947 hybrid_clock.cc:648] HybridClock initialized: now 1786344586177926 us; error 52 us; skew 500 ppm
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:46.186277 9947 init.cc:377] Logged in from keytab as kudu/127.15.232.254@KRBTEST.COM (short username kudu)
I20260810 06:49:46.186858 9947 webserver.cc:460] Webserver started at http://127.15.232.254:40847/ using document root <none> and password file <none>
I20260810 06:49:46.187122 9947 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:46.187173 9947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:46.187274 9947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:46.188366 9947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/data/instance:
uuid: "d4bb88655b8a432cafa802b63e53b22a"
format_stamp: "Formatted at 2026-08-10 06:49:46 on dist-test-slave-vxj2"
server_key: "7d6b8fcfa5acd0a8e224073f5db8a889"
server_key_iv: "bb635b070f8ba5a8b6089453c781af2f"
server_key_version: "encryptionkey@0"
I20260810 06:49:46.188699 9947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/wal/instance:
uuid: "d4bb88655b8a432cafa802b63e53b22a"
format_stamp: "Formatted at 2026-08-10 06:49:46 on dist-test-slave-vxj2"
server_key: "7d6b8fcfa5acd0a8e224073f5db8a889"
server_key_iv: "bb635b070f8ba5a8b6089453c781af2f"
server_key_version: "encryptionkey@0"
I20260810 06:49:46.190512 9947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:49:46.191422 9964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:46.191702 9947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:49:46.191896 9947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/wal
uuid: "d4bb88655b8a432cafa802b63e53b22a"
format_stamp: "Formatted at 2026-08-10 06:49:46 on dist-test-slave-vxj2"
server_key: "7d6b8fcfa5acd0a8e224073f5db8a889"
server_key_iv: "bb635b070f8ba5a8b6089453c781af2f"
server_key_version: "encryptionkey@0"
I20260810 06:49:46.191974 9947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:46.212250 9947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:46.214726 9947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:46.214955 9947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:46.219651 9947 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.254:34119
I20260810 06:49:46.219692 10023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.254:34119 every 8 connection(s)
I20260810 06:49:46.220126 9947 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:46.221088 10024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:46.222826 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 9947
I20260810 06:49:46.222947 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-0/wal/instance
I20260810 06:49:46.223129 16291 external_mini_cluster.cc:1428] Setting key 5741a5e58f86fa82c80e2d15779282a3
I20260810 06:49:46.223101 10024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.253@KRBTEST.COM" created.
I20260810 06:49:46.236135 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.224080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49294 (local address 127.15.232.254:34119)
0810 06:49:46.224310 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.224318 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.224464 (+ 146us) server_negotiation.cc:407] Connection header received
0810 06:49:46.225239 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.225242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.225271 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.225399 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.225993 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.227558 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.228863 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.229566 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.229710 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.232287 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.232298 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.232299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.232308 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.234294 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.234729 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.234732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.234733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.234821 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.235095 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.235096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.235096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.235368 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.235445 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.235685 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.235814 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":185,"thread_start_us":81,"threads_started":1}
I20260810 06:49:46.236155 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.224009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34119 (local address 127.0.0.1:49294)
0810 06:49:46.224250 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.224304 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:46.225003 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:46.225421 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:46.225424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:46.225693 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:46.225871 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.225876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.227730 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.227732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.228694 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.228698 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.229669 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.229670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.229736 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.230323 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:46.230332 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:46.232182 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:46.234381 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.234390 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.234392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.234643 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.234927 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.234927 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.234928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.235026 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.235455 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:46.235460 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:46.235525 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 06:49:46.235605 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:49:46.235918 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":114,"thread_start_us":102,"threads_started":1}
I20260810 06:49:46.237092 10024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:46.237640 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.237504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:43719 (local address 127.0.0.1:52134)
0810 06:49:46.237564 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.237616 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:43719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 06:49:46.237792 9966 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.253:43719: connect: Connection refused (error 111)
I20260810 06:49:46.238252 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.238149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:43719 (local address 127.0.0.1:52140)
0810 06:49:46.238199 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.238227 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:43719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 06:49:46.239486 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.253:43719: Network error: Client connection negotiation failed: client connection to 127.15.232.253:43719: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.253.keytab.
Entry for principal kudu/127.15.232.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.253.keytab.
I20260810 06:49:46.256259 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.253:43719
--webserver_interface=127.15.232.253
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.253.keytab
--principal=kudu/127.15.232.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:41859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:34119,127.15.232.253:43719,127.15.232.252:38889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:49:46.288781 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } attempt: 1
I20260810 06:49:46.289359 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.289225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:43719 (local address 127.0.0.1:52146)
0810 06:49:46.289285 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.289326 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:43719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 06:49:46.289722 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.289620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:43719 (local address 127.0.0.1:52156)
0810 06:49:46.289670 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.289702 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:43719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 06:49:46.289814 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.253:43719: Network error: Client connection negotiation failed: client connection to 127.15.232.253:43719: connect: Connection refused (error 111)
W20260810 06:49:46.346202 10035 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:49:46.346539 10035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:46.346589 10035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:46.346635 10035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:46.347960 10035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:46.348052 10035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:46.348081 10035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:46.348122 10035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:46.348157 10035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:46.349895 10035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:41859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:34119,127.15.232.253:43719,127.15.232.252:38889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.253.keytab
--principal=kudu/127.15.232.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.253:43719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.232.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:46.350349 10035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:46.350656 10035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:46.356195 10046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:46.356170 10041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:46.356262 10035 server_base.cc:1034] running on GCE node
W20260810 06:49:46.356170 10043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:46.356681 10035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:46.356971 10035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:46.357951 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } attempt: 2
I20260810 06:49:46.358165 10035 hybrid_clock.cc:648] HybridClock initialized: now 1786344586358158 us; error 67 us; skew 500 ppm
I20260810 06:49:46.358472 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.358326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:43719 (local address 127.0.0.1:52158)
0810 06:49:46.358392 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.358435 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:43719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 06:49:46.358844 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.358722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:43719 (local address 127.0.0.1:52160)
0810 06:49:46.358781 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.358811 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.253:43719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 06:49:46.359015 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.253:43719: Network error: Client connection negotiation failed: client connection to 127.15.232.253:43719: connect: Connection refused (error 111)
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:46.360982 10035 init.cc:377] Logged in from keytab as kudu/127.15.232.253@KRBTEST.COM (short username kudu)
I20260810 06:49:46.361471 10035 webserver.cc:460] Webserver started at http://127.15.232.253:41287/ using document root <none> and password file <none>
I20260810 06:49:46.361692 10035 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:46.361735 10035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:46.361831 10035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:46.363008 10035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/data/instance:
uuid: "38e5588415ea496693e3e895810d79a5"
format_stamp: "Formatted at 2026-08-10 06:49:46 on dist-test-slave-vxj2"
server_key: "2f301304c5baf75088cfff16177d4847"
server_key_iv: "af096bcc4e117a7e3b4b57921cdddc43"
server_key_version: "encryptionkey@0"
I20260810 06:49:46.363317 10035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/wal/instance:
uuid: "38e5588415ea496693e3e895810d79a5"
format_stamp: "Formatted at 2026-08-10 06:49:46 on dist-test-slave-vxj2"
server_key: "2f301304c5baf75088cfff16177d4847"
server_key_iv: "af096bcc4e117a7e3b4b57921cdddc43"
server_key_version: "encryptionkey@0"
I20260810 06:49:46.364677 10035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:46.365543 10053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:46.365757 10035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:49:46.365849 10035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/wal
uuid: "38e5588415ea496693e3e895810d79a5"
format_stamp: "Formatted at 2026-08-10 06:49:46 on dist-test-slave-vxj2"
server_key: "2f301304c5baf75088cfff16177d4847"
server_key_iv: "af096bcc4e117a7e3b4b57921cdddc43"
server_key_version: "encryptionkey@0"
I20260810 06:49:46.365916 10035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:46.403138 10035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:46.405748 10035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:46.405889 10035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:46.409987 10035 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.253:43719
I20260810 06:49:46.410027 10119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.253:43719 every 8 connection(s)
I20260810 06:49:46.410435 10035 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/data/info.pb
I20260810 06:49:46.411484 10120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:46.412878 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 10035
I20260810 06:49:46.412978 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-1/wal/instance
I20260810 06:49:46.413226 16291 external_mini_cluster.cc:1428] Setting key 051a392eef90dd7aa2e5d53c3d57626d
I20260810 06:49:46.413297 10120 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.253@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.252@KRBTEST.COM; defaulting to no policy
I20260810 06:49:46.424145 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.414218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49296 (local address 127.15.232.254:34119)
0810 06:49:46.414291 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.414294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.414561 (+ 267us) server_negotiation.cc:407] Connection header received
0810 06:49:46.415209 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.415211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.415229 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.415335 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.415986 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.416786 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.417880 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.418260 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.418353 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.420737 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.420739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.420739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.420751 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.422582 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.423101 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.423103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.423103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.423171 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.423440 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.423441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.423441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.423573 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.423636 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.423908 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.423989 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:49:46.424286 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.414188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34119 (local address 127.0.0.1:49296)
0810 06:49:46.414410 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.414464 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:46.415082 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:46.415418 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:46.415422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:46.415685 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:46.415890 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.415894 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.416894 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.416895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.417788 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.417791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.418385 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.418386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.418501 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.419153 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:46.419162 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:46.420629 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:46.422682 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.422691 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.422693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.423022 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.423262 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.423262 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.423263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.423377 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.423633 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:46.423638 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:46.423763 (+ 125us) client_negotiation.cc:769] Sending connection context
0810 06:49:46.423852 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:49:46.424039 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":106,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.252@KRBTEST.COM" created.
I20260810 06:49:46.424957 10120 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.253@KRBTEST.COM for kudu/127.15.232.253@KRBTEST.COM
I20260810 06:49:46.432753 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.425273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:43719 (local address 127.0.0.1:52174)
0810 06:49:46.425323 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.425334 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:46.425392 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:46.425667 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:46.425668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:46.425829 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:46.425880 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.425882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.427054 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.427055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.427568 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.427570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.428092 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.428093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.428161 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.428606 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:46.428611 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:46.429889 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:46.431686 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.431687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.431688 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.431862 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.432065 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.432066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.432067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.432124 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.432423 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:46.432424 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:46.432464 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 06:49:46.432529 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 06:49:46.432635 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 06:49:46.432819 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.425373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52174 (local address 127.15.232.253:43719)
0810 06:49:46.425537 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.425541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.425558 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:49:46.425574 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.425576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.425586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.425667 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.425963 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.426962 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.427643 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.428003 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.428085 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.429975 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.429985 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.429985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.429992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.431585 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.431941 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.431944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.431945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.432001 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.432191 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.432193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.432193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.432348 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.432447 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.432559 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.432691 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260810 06:49:46.433526 10120 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:46.433933 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.433777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44462)
0810 06:49:46.433810 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.433911 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260810 06:49:46.434082 10056 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
I20260810 06:49:46.434553 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.434450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44476)
0810 06:49:46.434497 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.434523 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 06:49:46.435930 10120 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:38889: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.252.keytab.
Entry for principal kudu/127.15.232.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.252.keytab.
I20260810 06:49:46.442286 16291 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpUSKt/build/release/bin/kudu
/tmp/dist-test-tasktpUSKt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.252:38889
--webserver_interface=127.15.232.252
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.252.keytab
--principal=kudu/127.15.232.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.212:41859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.254:34119,127.15.232.253:43719,127.15.232.252:38889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:49:46.450176 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } attempt: 3
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.253@KRBTEST.COM
I20260810 06:49:46.458077 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.450633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52176 (local address 127.15.232.253:43719)
0810 06:49:46.450702 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.450705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.450717 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:49:46.450759 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.450761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.450774 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.450865 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.451273 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.452119 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.452859 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.453258 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.453335 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.455429 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.455431 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.455431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.455438 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.456947 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.457367 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.457369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.457369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.457428 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.457619 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.457619 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.457619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.457724 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.457771 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.457937 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.458006 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:49:46.458137 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.450531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:43719 (local address 127.0.0.1:52176)
0810 06:49:46.450595 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.450610 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:46.450661 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:46.450943 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:46.450945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:46.451108 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:46.451180 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.451182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.452214 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.452215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.452767 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.452769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.453335 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.453337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.453444 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.453954 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:46.453959 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:46.455341 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:46.457031 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.457032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.457033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.457292 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.457492 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.457492 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.457493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.457561 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.457778 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:46.457779 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:46.457835 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 06:49:46.457919 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 06:49:46.458029 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:49:46.458694 10024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:46.459074 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.458966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44488)
0810 06:49:46.459017 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.459050 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 06:49:46.459470 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.459355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44498)
0810 06:49:46.459422 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.459447 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:49:46.459553 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:38889: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
I20260810 06:49:46.485234 10120 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } attempt: 1
I20260810 06:49:46.485841 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.485685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44512)
0810 06:49:46.485757 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.485801 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 06:49:46.486284 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.486182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44518)
0810 06:49:46.486235 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.486265 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 06:49:46.486413 10120 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:38889: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
I20260810 06:49:46.490707 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } attempt: 1
I20260810 06:49:46.491184 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.491053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44528)
0810 06:49:46.491116 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.491155 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 06:49:46.491650 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.491563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44536)
0810 06:49:46.491609 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.491634 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 06:49:46.491770 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:38889: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
W20260810 06:49:46.534188 10129 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:49:46.534490 10129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:46.534521 10129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:46.534567 10129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:46.535789 10129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:46.535861 10129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:46.535884 10129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:46.535907 10129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:46.535936 10129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:46.537588 10129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.212:41859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.254:34119,127.15.232.253:43719,127.15.232.252:38889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/krb5kdc/kudu_127.15.232.252.keytab
--principal=kudu/127.15.232.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.252:38889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.232.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:49:46.537885 10129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:46.538146 10129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:46.543401 10137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:46.543478 10136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:46.543390 10139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:46.543610 10129 server_base.cc:1034] running on GCE node
I20260810 06:49:46.543862 10129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:46.544142 10129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:46.545326 10129 hybrid_clock.cc:648] HybridClock initialized: now 1786344586545316 us; error 50 us; skew 500 ppm
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:46.548362 10129 init.cc:377] Logged in from keytab as kudu/127.15.232.252@KRBTEST.COM (short username kudu)
I20260810 06:49:46.548843 10129 webserver.cc:460] Webserver started at http://127.15.232.252:33009/ using document root <none> and password file <none>
I20260810 06:49:46.549203 10129 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:46.549292 10129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:46.549459 10129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:46.550645 10129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/data/instance:
uuid: "8450d5fcafa746d7983647998a167380"
format_stamp: "Formatted at 2026-08-10 06:49:46 on dist-test-slave-vxj2"
server_key: "8bb00c9d2bcac579e8f3c8b8b262bf7a"
server_key_iv: "c8cbe9536fe8941c6fcc29b7c275d0f6"
server_key_version: "encryptionkey@0"
I20260810 06:49:46.551060 10129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/wal/instance:
uuid: "8450d5fcafa746d7983647998a167380"
format_stamp: "Formatted at 2026-08-10 06:49:46 on dist-test-slave-vxj2"
server_key: "8bb00c9d2bcac579e8f3c8b8b262bf7a"
server_key_iv: "c8cbe9536fe8941c6fcc29b7c275d0f6"
server_key_version: "encryptionkey@0"
I20260810 06:49:46.552747 10129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:49:46.553737 10146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:46.554055 10129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:49:46.554177 10129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/wal
uuid: "8450d5fcafa746d7983647998a167380"
format_stamp: "Formatted at 2026-08-10 06:49:46 on dist-test-slave-vxj2"
server_key: "8bb00c9d2bcac579e8f3c8b8b262bf7a"
server_key_iv: "c8cbe9536fe8941c6fcc29b7c275d0f6"
server_key_version: "encryptionkey@0"
I20260810 06:49:46.554270 10129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:46.554574 10120 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } attempt: 2
I20260810 06:49:46.555012 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.554870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44548)
0810 06:49:46.554937 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.554979 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 06:49:46.555398 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.555304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44554)
0810 06:49:46.555347 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.555375 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 06:49:46.555516 10120 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:38889: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
I20260810 06:49:46.564199 10129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:46.566476 10129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:46.566651 10129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:46.566920 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } attempt: 2
I20260810 06:49:46.567404 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.567214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44556)
0810 06:49:46.567292 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.567341 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 06:49:46.567817 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.567709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44564)
0810 06:49:46.567771 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.567798 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 06:49:46.567947 10024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.252:38889: Network error: Client connection negotiation failed: client connection to 127.15.232.252:38889: connect: Connection refused (error 111)
I20260810 06:49:46.571671 10129 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.252:38889
I20260810 06:49:46.571707 10208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.252:38889 every 8 connection(s)
I20260810 06:49:46.572093 10129 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/data/info.pb
I20260810 06:49:46.573087 10210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:46.574761 10210 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:46.578214 16291 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpUSKt/build/release/bin/kudu as pid 10129
I20260810 06:49:46.578302 16291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpUSKt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344303981681-16291-0/minicluster-data/master-2/wal/instance
I20260810 06:49:46.578531 16291 external_mini_cluster.cc:1428] Setting key a19a26b701e0ef53c2d9e29298489550
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.252@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.252@KRBTEST.COM
I20260810 06:49:46.585659 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.575530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49312 (local address 127.15.232.254:34119)
0810 06:49:46.575596 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.575599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.575803 (+ 204us) server_negotiation.cc:407] Connection header received
0810 06:49:46.576403 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.576405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.576418 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.576478 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.577247 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.577720 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.578954 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.579271 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.579341 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.582131 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.582133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.582133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.582141 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.584006 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.584454 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.584456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.584456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.584527 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.584858 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.584859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.584859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.584973 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.585028 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.585447 (+ 419us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.585543 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:49:46.585744 10212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.575486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.254:34119 (local address 127.0.0.1:49312)
0810 06:49:46.575663 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.575709 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:46.576273 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:46.576557 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:46.576561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:46.576842 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:46.577048 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.577052 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.577849 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.577851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.578862 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.578865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.579362 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.579363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.579509 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.580069 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:46.580080 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:46.582004 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:46.584102 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.584109 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.584110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.584359 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.584615 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.584616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.584616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.584775 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.585045 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:46.585052 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:46.585276 (+ 224us) client_negotiation.cc:769] Sending connection context
0810 06:49:46.585393 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:49:46.585526 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260810 06:49:46.586694 10210 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:46.589063 10214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.578980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.15.232.252:38889)
0810 06:49:46.579145 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.579149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.579193 (+ 44us) server_negotiation.cc:407] Connection header received
0810 06:49:46.579223 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.579225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.579245 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.579340 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.580532 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.581641 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.582501 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.582913 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.582976 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.585706 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.585712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.585713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.585721 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.587838 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.588203 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.588206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.588207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.588270 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.588487 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.588487 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.588488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.588633 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.588818 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.588861 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.588943 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":78,"threads_started":1}
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.252@KRBTEST.COM for kudu/127.15.232.253@KRBTEST.COM
I20260810 06:49:46.594048 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.587056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.15.232.253:43719)
0810 06:49:46.587134 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.587136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.587148 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:49:46.587232 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.587233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.587242 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.587327 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.587596 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.588084 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.588773 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.588997 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.589094 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.591353 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.591355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.591355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.591364 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.592893 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.593314 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.593315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.593316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.593366 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.593575 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.593576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.593576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.593670 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.593716 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.593900 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.593983 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:49:46.594129 10212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.586982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.253:43719 (local address 127.0.0.1:52186)
0810 06:49:46.587107 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.587119 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:46.587166 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:46.587343 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:46.587344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:46.587481 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:46.587521 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.587522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.588177 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.588178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.588680 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.588682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.589089 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.589090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.589296 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.589807 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:46.589811 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:46.591269 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:46.593001 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.593002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.593002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.593255 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.593442 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.593443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.593444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.593510 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.593739 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:46.593740 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:46.593794 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 06:49:46.593885 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 06:49:46.593992 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260810 06:49:46.594816 10210 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.254@KRBTEST.COM
I20260810 06:49:46.598744 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.589993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.15.232.254:34119)
0810 06:49:46.590055 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.590056 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.590065 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:49:46.590088 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.590089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.590099 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.590160 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.590769 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.591341 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.592264 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.592579 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.592654 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.595554 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.595555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.595556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.595564 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.597284 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.597671 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.597673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.597673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.597751 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.598120 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.598122 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.598123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.598291 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.598512 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.598555 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.598645 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.252@KRBTEST.COM for kudu/127.15.232.252@KRBTEST.COM
I20260810 06:49:46.602415 10212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.595117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44590)
0810 06:49:46.595151 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.595163 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:46.595202 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:46.595332 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:46.595332 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:46.595472 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:46.595524 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.595526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.596208 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.596209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.597015 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.597019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.597668 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.597669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.597733 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.598169 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:46.598174 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:46.599678 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:46.601416 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.601417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.601418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.601664 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.601856 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.601857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.601857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.601920 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.602187 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:46.602188 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:46.602224 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:49:46.602278 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 06:49:46.602350 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260810 06:49:46.602545 10214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.595209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44590 (local address 127.15.232.252:38889)
0810 06:49:46.595244 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.595246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.595253 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:49:46.595268 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.595269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.595278 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.595320 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.595608 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.596122 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.597140 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.597533 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.597615 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.599760 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.599761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.599762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.599768 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.601343 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.601740 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.601741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.601741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.601791 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.601982 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.601983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.601983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.602118 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.602183 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.602319 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.602450 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":20}
I20260810 06:49:46.604063 10210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380: Bootstrap starting.
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.253@KRBTEST.COM
I20260810 06:49:46.605055 10210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:46.605437 10210 log.cc:824] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:46.606401 10210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380: No bootstrap required, opened a new log
I20260810 06:49:46.607589 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.599491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.15.232.253:43719)
0810 06:49:46.599530 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.599532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.599540 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:49:46.599599 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.599600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.599608 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.599653 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.600286 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.601046 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.602088 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.602395 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.602468 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.604822 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.604824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.604824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.604833 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.606383 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.606755 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.606756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.606757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.606820 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.607038 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.607038 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.607039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.607154 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.607368 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.607409 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.607485 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:49:46.608191 10210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } }
I20260810 06:49:46.608366 10210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8450d5fcafa746d7983647998a167380, State: Initialized, Role: FOLLOWER
I20260810 06:49:46.608459 10210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } }
I20260810 06:49:46.608757 10216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } } }
I20260810 06:49:46.608835 10210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:46.608855 10216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:46.609342 16291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 06:49:46.610702 10227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:49:46.610759 10227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:49:46.610631 9995 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d4bb88655b8a432cafa802b63e53b22a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49316
I20260810 06:49:46.611133 10129 master_runner.cc:186] Error getting master registration for 127.15.232.254:34119: OK, instance_id { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" instance_seqno: 1786344586218331 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:49:46.614212 10129 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:49:46.646687 10120 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } attempt: 3
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.253@KRBTEST.COM for kudu/127.15.232.252@KRBTEST.COM
I20260810 06:49:46.656355 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.647109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44604)
0810 06:49:46.647193 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.647210 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:46.647277 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:46.647514 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:46.647516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:46.647729 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:46.647788 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.647790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.648961 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.648963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.650068 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.650071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.650606 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.650607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.650672 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.651221 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:46.651227 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:46.652807 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:46.654763 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.654765 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.654765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.655045 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.655301 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.655303 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.655303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.655394 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.655996 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:46.655997 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:46.656064 (+ 67us) client_negotiation.cc:769] Sending connection context
0810 06:49:46.656136 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 06:49:46.656249 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 06:49:46.656440 10214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.647194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.15.232.252:38889)
0810 06:49:46.647363 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.647365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.647384 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:49:46.647413 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.647414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.647425 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.647480 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.647892 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.648820 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.650169 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.650515 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.650595 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.652915 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.652917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.652917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.652927 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.654644 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.655126 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.655127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.655127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.655195 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.655480 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.655481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.655482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.655753 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.655839 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.656187 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.656294 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260810 06:49:46.658908 10120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5: Bootstrap starting.
I20260810 06:49:46.659716 10120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:46.660038 10120 log.cc:824] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:46.660810 10120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5: No bootstrap required, opened a new log
I20260810 06:49:46.662302 10120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } }
I20260810 06:49:46.662423 10120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38e5588415ea496693e3e895810d79a5, State: Initialized, Role: FOLLOWER
I20260810 06:49:46.662478 10120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } }
I20260810 06:49:46.662738 10230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } } }
I20260810 06:49:46.662817 10120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:46.662832 10230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:49:46.664700 10241 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:49:46.664762 10241 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:49:46.667088 10024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } attempt: 3
I20260810 06:49:46.664947 10035 master_runner.cc:186] Error getting master registration for 127.15.232.254:34119: OK, instance_id { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" instance_seqno: 1786344586218331 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:49:46.668005 10035 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 10 06:49:46 dist-test-slave-vxj2 krb5kdc[9932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344586, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.254@KRBTEST.COM for kudu/127.15.232.252@KRBTEST.COM
I20260810 06:49:46.675391 10214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.667633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44612 (local address 127.15.232.252:38889)
0810 06:49:46.667676 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.667678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.667685 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:49:46.667711 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.667712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.667728 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.667780 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.668232 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.668985 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.670004 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.670320 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.670381 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.672482 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.672484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.672484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.672492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.674231 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.674579 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.674581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.674581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.674640 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.674836 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.674836 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.674837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.674979 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.675042 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.675213 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.675292 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:49:46.675400 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.667379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.252:38889 (local address 127.0.0.1:44612)
0810 06:49:46.667432 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:46.667443 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:46.667484 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:46.667837 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:46.667838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:46.668081 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:46.668135 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.668137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.669084 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.669086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.669925 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:46.669927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.670386 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.670387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:46.670444 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.670950 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:46.670956 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:46.672399 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:46.674302 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.674304 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.674304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.674496 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.674713 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:46.674713 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:46.674714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:46.674773 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:46.675034 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:46.675035 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:46.675093 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 06:49:46.675186 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 06:49:46.675292 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 06:49:46.677068 10024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a: Bootstrap starting.
I20260810 06:49:46.677886 10024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:46.678191 10024 log.cc:824] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:46.679067 10024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a: No bootstrap required, opened a new log
I20260810 06:49:46.680428 10024 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } }
I20260810 06:49:46.680574 10024 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb88655b8a432cafa802b63e53b22a, State: Initialized, Role: FOLLOWER
I20260810 06:49:46.680653 10024 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } }
I20260810 06:49:46.680977 10024 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:46.681016 9995 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 06:49:46.681066 9995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:46.682005 9995 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } }
I20260810 06:49:46.682825 10085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d4bb88655b8a432cafa802b63e53b22a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "38e5588415ea496693e3e895810d79a5"
I20260810 06:49:46.682981 10085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [term 0 FOLLOWER]: Advancing to term 1
W20260810 06:49:46.683010 10258 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:49:46.683049 10258 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:49:46.682304 9995 leader_election.cc:288] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [CANDIDATE]: Term 1 election: Requested vote from peers 38e5588415ea496693e3e895810d79a5 (127.15.232.253:43719), 8450d5fcafa746d7983647998a167380 (127.15.232.252:38889)
I20260810 06:49:46.683969 10085 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb88655b8a432cafa802b63e53b22a in term 1.
I20260810 06:49:46.684281 10177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d4bb88655b8a432cafa802b63e53b22a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8450d5fcafa746d7983647998a167380"
I20260810 06:49:46.684437 10177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:46.685279 10177 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb88655b8a432cafa802b63e53b22a in term 1.
I20260810 06:49:46.685547 9967 leader_election.cc:302] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8450d5fcafa746d7983647998a167380, d4bb88655b8a432cafa802b63e53b22a; no voters:
I20260810 06:49:46.687291 9947 master_runner.cc:263] No leader master found from master d4bb88655b8a432cafa802b63e53b22a
I20260810 06:49:46.687347 9947 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:49:46.680945 10245 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } } }
I20260810 06:49:46.687472 10245 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:46.687527 10245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:46.687577 10245 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [term 1 LEADER]: Becoming Leader. State: Replica: d4bb88655b8a432cafa802b63e53b22a, State: Running, Role: LEADER
I20260810 06:49:46.687659 10245 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } }
I20260810 06:49:46.688115 10245 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4bb88655b8a432cafa802b63e53b22a. Latest consensus state: current_term: 1 leader_uuid: "d4bb88655b8a432cafa802b63e53b22a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } } }
I20260810 06:49:46.688198 10245 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:46.688416 10267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:46.688601 10267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:46.696332 10177 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [term 1 FOLLOWER]: Refusing update from remote peer d4bb88655b8a432cafa802b63e53b22a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:46.696719 10264 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:46.698632 10085 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [term 1 FOLLOWER]: Refusing update from remote peer d4bb88655b8a432cafa802b63e53b22a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:46.698921 10264 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [LEADER]: Connected to new peer: Peer: permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:46.699213 10216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4bb88655b8a432cafa802b63e53b22a. Latest consensus state: current_term: 1 leader_uuid: "d4bb88655b8a432cafa802b63e53b22a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } } }
I20260810 06:49:46.699308 10216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:46.700438 10214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.688617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44618 (local address 127.15.232.252:38889)
0810 06:49:46.688661 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.688664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.691720 (+ 3056us) server_negotiation.cc:407] Connection header received
0810 06:49:46.691836 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.691838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.691849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.691884 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.692726 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.693527 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.694658 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.694971 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.695013 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.697088 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.697089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.697089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.697096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.698868 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.699289 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.699290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.699290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.699366 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.699773 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.699774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.699775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.699912 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.699994 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.700215 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.700305 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:49:46.700632 10230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4bb88655b8a432cafa802b63e53b22a. Latest consensus state: current_term: 1 leader_uuid: "d4bb88655b8a432cafa802b63e53b22a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } } }
I20260810 06:49:46.700745 10230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:46.706358 10264 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4bb88655b8a432cafa802b63e53b22a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } } }
I20260810 06:49:46.706516 10264 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:46.707108 10268 mvcc.cc:205] Tried to move back new op lower bound from 7316867427099095040 to 7316867427073454080. Current Snapshot: MvccSnapshot[applied={T|T < 7316867427099095040}]
I20260810 06:49:46.707252 10216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4bb88655b8a432cafa802b63e53b22a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } } }
I20260810 06:49:46.707401 10216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:46.708480 10264 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4bb88655b8a432cafa802b63e53b22a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } } }
I20260810 06:49:46.708588 10264 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:46.708956 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.691772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49322 (local address 127.15.232.254:34119)
0810 06:49:46.691815 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.691817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.691996 (+ 179us) server_negotiation.cc:407] Connection header received
0810 06:49:46.692088 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.692090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.692099 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.692129 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.692793 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.693516 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.695208 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.695481 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.695523 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.705341 (+ 9818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.705343 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.705343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.705362 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.707244 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.707720 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.707721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.707721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.707778 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.707980 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.707981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.707981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.708373 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.708465 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.708760 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.708846 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:49:46.709076 10230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4bb88655b8a432cafa802b63e53b22a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4bb88655b8a432cafa802b63e53b22a" member_type: VOTER last_known_addr { host: "127.15.232.254" port: 34119 } } peers { permanent_uuid: "38e5588415ea496693e3e895810d79a5" member_type: VOTER last_known_addr { host: "127.15.232.253" port: 43719 } } peers { permanent_uuid: "8450d5fcafa746d7983647998a167380" member_type: VOTER last_known_addr { host: "127.15.232.252" port: 38889 } } }
I20260810 06:49:46.709275 10274 mvcc.cc:205] Tried to move back new op lower bound from 7316867427099095040 to 7316867427073454080. Current Snapshot: MvccSnapshot[applied={T|T < 7316867427099095040}]
I20260810 06:49:46.709295 10230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:46.710408 10267 catalog_manager.cc:1347] Generated new cluster ID: 74fa8036680546bc8c87a773cd60c0d8
I20260810 06:49:46.710461 10267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:46.710713 10272 mvcc.cc:205] Tried to move back new op lower bound from 7316867427099095040 to 7316867427073454080. Current Snapshot: MvccSnapshot[applied={T|T < 7316867427099095040}]
I20260810 06:49:46.730800 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:46.691978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52212 (local address 127.15.232.253:43719)
0810 06:49:46.692033 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.692035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.696649 (+ 4614us) server_negotiation.cc:407] Connection header received
0810 06:49:46.696761 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.696762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.696775 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.696807 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.698162 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.698877 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.706712 (+ 7835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:46.707050 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:46.707115 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:46.709947 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:46.709948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:46.709949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:46.709956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:46.712515 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.729739 (+ 17224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.729741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.729742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.729823 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:46.730047 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:46.730047 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:46.730048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:46.730205 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:46.730298 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:46.730584 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 06:49:46.730699 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:49:46.750558 10267 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:46.751543 10267 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:46.764202 10267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4bb88655b8a432cafa802b63e53b22a: Generated new TSK 0
I20260810 06:49:46.764498 10267 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:49:46.791531 10214 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:46.789626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44626 (local address 127.15.232.252:38889)
0810 06:49:46.789679 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.789682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.790784 (+ 1102us) server_negotiation.cc:407] Connection header received
0810 06:49:46.790902 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.790904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.790916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.790950 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.791456 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44626: BlockingRecv error: recv got EOF from 127.0.0.1:44626 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260810 06:49:46.792742 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:46.790106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49332 (local address 127.15.232.254:34119)
0810 06:49:46.790152 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.790154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.791969 (+ 1815us) server_negotiation.cc:407] Connection header received
0810 06:49:46.792196 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.792197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.792211 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.792249 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.792654 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49332: BlockingRecv error: recv got EOF from 127.0.0.1:49332 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 06:49:46.793604 10126 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:46.790453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52214 (local address 127.15.232.253:43719)
0810 06:49:46.790499 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.790501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.792134 (+ 1633us) server_negotiation.cc:407] Connection header received
0810 06:49:46.793014 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.793016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.793030 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.793091 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.793535 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52214: BlockingRecv error: recv got EOF from 127.0.0.1:52214 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 06:49:46.795681 10126 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:46.794774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52216 (local address 127.15.232.253:43719)
0810 06:49:46.794831 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.794833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.794966 (+ 133us) server_negotiation.cc:407] Connection header received
0810 06:49:46.795216 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.795217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.795237 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.795318 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.795632 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52216: BlockingRecv error: recv got EOF from 127.0.0.1:52216 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260810 06:49:46.798367 10126 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:46.797831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52222 (local address 127.15.232.253:43719)
0810 06:49:46.797880 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.797882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.797891 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:49:46.797953 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.797954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.797964 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.798010 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.798293 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52222: BlockingRecv error: recv got EOF from 127.0.0.1:52222 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 06:49:46.802129 10126 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:46.801428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.15.232.253:43719)
0810 06:49:46.801474 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.801477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.801628 (+ 151us) server_negotiation.cc:407] Connection header received
0810 06:49:46.801678 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.801679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.801689 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.801738 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.802076 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52234: BlockingRecv error: recv got EOF from 127.0.0.1:52234 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 06:49:46.808151 10126 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:46.807404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52250 (local address 127.15.232.253:43719)
0810 06:49:46.807450 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.807452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.807604 (+ 152us) server_negotiation.cc:407] Connection header received
0810 06:49:46.807686 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.807687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.807701 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.807752 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.808080 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52250: BlockingRecv error: recv got EOF from 127.0.0.1:52250 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 06:49:46.818465 10126 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:46.817346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52264 (local address 127.15.232.253:43719)
0810 06:49:46.817397 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.817400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.817827 (+ 427us) server_negotiation.cc:407] Connection header received
0810 06:49:46.817961 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.817962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.817980 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.818047 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.818405 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52264: BlockingRecv error: recv got EOF from 127.0.0.1:52264 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 06:49:46.836440 10126 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:46.835592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52278 (local address 127.15.232.253:43719)
0810 06:49:46.835642 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.835645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.835815 (+ 170us) server_negotiation.cc:407] Connection header received
0810 06:49:46.835911 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.835912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.835924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.835997 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.836352 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52278: BlockingRecv error: recv got EOF from 127.0.0.1:52278 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 06:49:46.870985 10126 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:46.870176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.15.232.253:43719)
0810 06:49:46.870251 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.870254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.870405 (+ 151us) server_negotiation.cc:407] Connection header received
0810 06:49:46.870495 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.870496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.870508 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.870561 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.870920 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52292: BlockingRecv error: recv got EOF from 127.0.0.1:52292 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 06:49:46.936933 10126 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:46.936207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52298 (local address 127.15.232.253:43719)
0810 06:49:46.936309 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:46.936311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:46.936326 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:49:46.936406 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:46.936407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:46.936426 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:46.936481 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:46.936855 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52298: BlockingRecv error: recv got EOF from 127.0.0.1:52298 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 06:49:47.066946 10126 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:47.066230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52308 (local address 127.15.232.253:43719)
0810 06:49:47.066316 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:47.066318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:47.066385 (+ 67us) server_negotiation.cc:407] Connection header received
0810 06:49:47.066441 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:47.066442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:47.066463 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:47.066529 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:47.066871 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52308: BlockingRecv error: recv got EOF from 127.0.0.1:52308 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 06:49:47.324977 10126 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:47.324315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.15.232.253:43719)
0810 06:49:47.324408 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:47.324416 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:47.324429 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:49:47.324493 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:47.324494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:47.324505 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:47.324573 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:47.324922 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52318: BlockingRecv error: recv got EOF from 127.0.0.1:52318 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:49:47.611212 10227 catalog_manager.cc:1259] Loaded cluster ID: 74fa8036680546bc8c87a773cd60c0d8
I20260810 06:49:47.611306 10227 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380: loading cluster ID for follower catalog manager: success
I20260810 06:49:47.612131 10227 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380: acquiring CA information for follower catalog manager: success
I20260810 06:49:47.612425 10227 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8450d5fcafa746d7983647998a167380: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:49:47.665297 10241 catalog_manager.cc:1259] Loaded cluster ID: 74fa8036680546bc8c87a773cd60c0d8
I20260810 06:49:47.665387 10241 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5: loading cluster ID for follower catalog manager: success
I20260810 06:49:47.666210 10241 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5: acquiring CA information for follower catalog manager: success
I20260810 06:49:47.666529 10241 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 38e5588415ea496693e3e895810d79a5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:49:47.841315 10355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:47.838612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52330 (local address 127.15.232.253:43719)
0810 06:49:47.838781 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:47.838783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:47.838793 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:49:47.838872 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:47.838873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:47.838939 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:47.839008 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:47.839156 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:47.839670 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:47.840472 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:47.840675 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:47.840747 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:47.840905 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:47.841046 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:47.841122 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:47.841173 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 06:49:47.841194 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":100,"threads_started":1}
I20260810 06:49:47.845772 10363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:47.843303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.15.232.252:38889)
0810 06:49:47.843424 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:47.843426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:47.843434 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:49:47.843483 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:47.843483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:47.843521 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:47.843597 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:47.843786 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:47.844265 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:47.844963 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:47.845187 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:47.845250 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:47.845401 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:47.845506 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:47.845566 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:47.845643 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 06:49:47.845665 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
I20260810 06:49:47.850780 10355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:47.847682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52344 (local address 127.15.232.253:43719)
0810 06:49:47.847739 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:47.847741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:47.847983 (+ 242us) server_negotiation.cc:407] Connection header received
0810 06:49:47.848106 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:47.848107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:47.848121 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:47.848173 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:47.848315 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:47.848799 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:47.849949 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:47.850249 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:47.850316 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:47.850471 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:47.850505 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:47.850558 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:47.850652 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 06:49:47.850678 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:49:47.851087 10373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:47.847717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49336 (local address 127.15.232.254:34119)
0810 06:49:47.847897 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:47.847899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:47.847910 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:49:47.847970 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:47.847971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:47.848027 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:47.848113 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:47.848372 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:47.849091 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:47.849887 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:47.850181 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:47.850262 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:47.850638 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:47.850753 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:47.850811 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:47.850987 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 06:49:47.851010 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":129,"threads_started":1}
I20260810 06:49:47.851441 10363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:47.847516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44634 (local address 127.15.232.252:38889)
0810 06:49:47.847556 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:47.847558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:47.847761 (+ 203us) server_negotiation.cc:407] Connection header received
0810 06:49:47.847881 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:47.847882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:47.847893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:47.847953 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:47.848064 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:47.848778 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:47.850347 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:47.850624 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:47.850682 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:47.850887 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:47.850924 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:47.850978 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:47.851326 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 06:49:47.851353 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 06:49:47.852852 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 9947
I20260810 06:49:47.860335 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 10035
I20260810 06:49:47.866575 16291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpUSKt/build/release/bin/kudu with pid 10129
2026-08-10T06:49:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3102 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6050 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (283895 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST